I recently got a grant rejection - but I swear this blog post won't be whinging about rejections. Rather, I'd like to start by sharing a reviewer's comment that surprised me: The proposal is not novel, they wrote, because it's all about replicability and open science and blah blah blah, and we already know all about that since the Ioannidis 2005 paper, published almost 20 years ago now! Yes, I replied to the reviewer in my head. But have we actually solved this issue?
In a way, I understand where the reviewer is coming from. The other day, I opened the latest issue of the German academic journal, "Forschung und Lehre". On the first page was an article about the p-value, and how it doesn't mean what many researchers seem to think it means. "But we've been talking about that for decades now, surely everyone already knows this!", I thought and skipped to the next page.
Today, I taught a workshop on Open Science for a masters programme. I've been doing similar courses for similar audiences for a number of years now. Every year, I show a slide with the results of the Open Science Collaboration (2015) replication efforts. "Who has heard of this study before?" I ask. I started teaching in about 2016, and found that most students, including bachelor students, were familiar with the study and its provocative results. Today, what I was presenting seemed to be new to many students. On the one hand, that's good for me - I was able to tell the students something new, rather than repeating things they already knew, anyway. On the other hand, I wondered, do people not care about replicability any more?
The Open Science community, at the beginning, was a close-knit group on twitter. My reputation in academia (such as it is) is largely thanks to this community: from the beginning, I was active by tweeting and writing blog posts about Open Science, and within the community, such posts were spread widely. However, long before this community was scattered across various alternative platforms such as Mastodon and BlueSky, it had grown into fractions that spent a lot of their time fighting each other. Fashions come and go - I have learned that in my teenage years, after which I made the conscious decision to ignore all clothing trends. So maybe Open Science is just not cool anymore.
This raises the question: Has the open science movement failed in it mission to improve science? Or, on the contrary, did it solve the issues so efficiently that it is no longer needed? The first scenario is, unfortunately, more likely. I myself am guilty of having been too dogmatic and over-simplifying, in my mind, the ways in which Open Science can, and should, improve science. But has Open Science really unleashed its full potential in improving science? I sincerely believe that this is not the case. I feel like the discussions about how Open Science works and, indeed, what outcome we want to achieve, is only just starting to take shape. Many questions remain, such as: What is important for good research? Via what mechanisms do Open Science practices impact the research quality (positively or negatively)?
This blog post, again, has more open questions than answers. So, dear Reviewer 2, if you're reading this blog post: When you review proposals involving reproducibility and Open Science, please don't reject them on the basis that we already know everything already.
