Some thoughts on trivial results, or: Yet another argument for Registered Reports

A senior colleague once joked: “If I read about a new result, my reaction is either: ‘That’s trivial!’, or ‘I don’t believe that!’”

These types of reactions are pretty common when presenting the results of a new study (in my experience, anyway). In peer review, especially the former can be a reason for a paper rejection. In conversations with colleagues, one sometimes gets told, jokingly: “Well, I could have told you in advance that you’d get this result, you didn’t have to run the study!” This can be quite discouraging, especially if, while you were planning your study, it did not seem at all obvious to you that you would get the obtained result.

In many cases, perhaps, the outcomes of a result are obvious, especially to someone who has been in the field for much longer than you are. For some effects, there might be huge file drawers, such that it’s a well-known secret that an experimental paradigm which seems perfectly reasonable at first sight doesn’t actually work. In this case, it would be very helpful to hear that it’s probably not the best idea to invest time and resources on this paradigm. However, it would be even more helpful to hear about this before you plan and execute your study.  

One also needs to take into account that there is hindsight bias. If you hear the results first, it’s easy to come up with an explanation for the exact obtained pattern. Thus, a some result that might seem trivial in hindsight would actually have been not so east to predict a priori. There is also often disagreement about the triviality of an outcome: It's not unheard of (not only in my experience) that Reviewer 1 claims that the paper shouldn't be published because the result is trivial, while Reviewer 2 recommends rejection because (s)he doesn’t believe this result.

Registered reports should strongly reduce the amount of times that people tell you that your results are trivial. If you submit a plan to do an experiment that really is trivial, the reviewers should point this out while evaluating the Stage 1 manuscript. If they have a good point, this will save you from collecting data for a study that many people might not find interesting. And if the reviewers agree that the research question is novel and interesting, they cannot later do a backflip and say that it’s trivial after having seen the results.

So, this is another advantage of registered reports. And, if I’m brave enough, I’ll change the way I tell (senior) colleagues about my work in informal conversations, from: “I did experiment X, and I got result Y” to “I did experiment X. What do you think happened?”