Recently I had an experience of an open peer review journal and it is certainly worth sharing.
As a student of behavior I have been thinking of the behavior of different players in the science publishing system and whether we can design a behaviorally sound system that would minimize biases , misconducts and irresponsible behaviors in the peer review system. I gave a talk on this in December 2017 and a preprint article in 2019. Interestingly in 2019 itself an open peer review journal called Qeios (pronounced like chaos) was started on very parallel (but not quite the same, some crucial components from my system missing) principles. Qeios started as a preprint repository but also has a peer reviewed publication system but with no dichotomous editorial decision involved. An AI system searches for and invites reviewers. Reviewers are informed that their reviews will be public. Authors can respond to them, improve their paper if needed but all these steps and stages will be completely transparent.
Any new journal has initial problems which this journal will also have for quite some time. Because of the impact factor illusion and the sheep mentality of researchers, a new journal is unlikely to get high quality papers in the initial years, which seems to be true for Qeios as expected. When I received a review request, I didn’t expect a high quality paper. What I received was not bad, it had some thought provoking ideas, but the work was not rigorous enough, being sort of aimless and not contributing any new and meaningful insights. At the same time it was Ok in terms of the model developed and some of the data and discussion.
Although I have been advocating open peer reviews for many years now, while preparing myself for an open peer review I realized something that I had failed to appreciate before. For a conventional journal I would have recommended rejection. I realized that it would have been a very dumb and non-productive end. The paper certainly had triggered some thinking in me. I thought of some new questions, some ideas, some tricky issues for the first time. It was not enough to make a full paper in itself, but it was worth something. If I expressed my doubts, my half baked ideas it would possibly stimulate someone else to think. They may have solutions that I couldn’t think of, or they might simply add more questions, may even point out that I was wrong. All this is a valuable process, not a product. But so far we were only publishing products and hiding the process. That is only half science. Open peer review journals can bring out at least part of the thinking process and much can be learnt from that. May be at a later stage I will return to my own arguments and develop them further. May be someone else does so. If I had recommended rejection, then nothing of this sort would have happened. Me too would have forgotten the issue in no time. Publishing the thinking process enriched me substantially. Of course not everyone will do this. There would be a lot of junk published. It is true that some people would only write goody -goody reviews and since it is published, add to their CV. This is happening. That might be inevitable garbage, but not a sufficient reason to block the valuable open thinking process. Here is the link to what I wrote as a reviewer (https://www.qeios.com/read/S390H3). The author responded to my comments, not with much rigor but that’s ok. Everyone has limitations. If it is in the open domain someone else can compensate for these limitations sometime and take the concept to a meaningful level. I hope people understand and realize the strengths of open peer review and adopt this practice increasingly. The starting problem of quality would vanish eventually.
My science, My way 