1 code implementation • 6 Mar 2024 • Håkon Hanisch Kjærnli, Lluis Mas-Ribas, Aida Ashrafi, Gleb Sizov, Helge Langseth, Odd Erik Gundersen
Because most of the training data does not reflect such changes, the models present poor performance on the new out-of-distribution scenarios and, therefore, the impact of such events cannot be reliably anticipated ahead of time.
no code implementations • 11 Dec 2023 • Kevin Coakley, Christine R. Kirkpatrick, Odd Erik Gundersen
To account for these implementation factors, researchers should run their experiments multiple times in different hardware and software environments to verify their conclusions are not affected.
no code implementations • 15 Aug 2023 • Sayash Kapoor, Emily Cantrell, Kenny Peng, Thanh Hien Pham, Christopher A. Bail, Odd Erik Gundersen, Jake M. Hofman, Jessica Hullman, Michael A. Lones, Momin M. Malik, Priyanka Nanayakkara, Russell A. Poldrack, Inioluwa Deborah Raji, Michael Roberts, Matthew J. Salganik, Marta Serra-Garcia, Brandon M. Stewart, Gilles Vandewiele, Arvind Narayanan
Machine learning (ML) methods are proliferating in scientific research.
no code implementations • 15 Apr 2022 • Odd Erik Gundersen, Kevin Coakley, Christine Kirkpatrick, Yolanda Gil
Objective: The objective of this paper is to develop a framework that enable applied data science practitioners and researchers to understand which experiment design choices can lead to false findings and how and by this help in analyzing the conclusions of reproducibility experiments.
no code implementations • 19 Nov 2020 • Odd Erik Gundersen
The scientific method is analysed and characterised in order to develop the terminology required to define reproducibility.