David Peterson at Nature: “Alarm about a ‘replication crisis’ launched a wave of projects that aimed to quantitatively evaluate scientific reproducibility: statistical analyses, mass replications and surveys. Such efforts, collectively called metascience, have grown into a social movement advocating broad reforms: open-science mandates, preregistration of experiments and new incentives for careful research. It has drawn attention to the need for improvements, and caused rancour.
Philosophers, historians and sociologists no longer accept a single, unified definition of science. Instead, they document how scientists in different fields have developed unique practices of producing, communicating and evaluating evidence, guided loosely by a set of shared values. However, this diversity and underlying scholarship are often overlooked by metascience activists.
Over the past three years, Aaron Panofsky, a sociologist at the University of California, Los Angeles, and I have interviewed 60 senior biologists, chemists, geologists and physicists who are reviewing editors at Science, plus another 83 scientists seeking science-wide reforms. These highly recognized researchers saw growing interest in making science more open and robust — but also expressed scepticism.
Senior researchers bristled at the idea that their fields were in ‘crisis’, and suspected that activists were seeking recognition for themselves. A frustrated biologist argued that people running mass replication studies “were not motivated to find reproducibility” and benefited from finding it lacking. Others said metascientists dismissed replication work done to further a line of research rather than assess the state of the literature. Another saw data deposition as a frustrating, externally imposed mandate: “We’re already drowning in all the bureaucratic crap.”
Even those who acknowledged the potential value of reforms, such as those for data sharing, felt that there was no discussion about the costs. “If you add up all of the things that only take ten minutes, it’s a huge chunk of your day.”…(More)”.
