Transparency of methods, data, analytical tools, programs used is extremely important for science. This is well recognized and many journals ask many questions to authors to ensure transparency even before processing a manuscript. Ironically peer reviews, which form the backbone of today’s science publishing, are not transparent and accessible to anyone. I haven’t seen a bigger contradiction in my life.
Recently I uploaded two papers on the open peer review journal Qeios. One titled “Behavioural optimization in scientific publishing (https://doi.org/10.32388/8W10ND.3)” and another more recently uploaded “Evolution of new variants during the SARS-Cov-2 pandemic: mutation limited or selection limited? (https://doi.org/10.32388/LLA6AO)”. The first one, in less than one month has received 13 reviews (in addition two elaborate responses to me on email) and has undergone two revisions. This response is unique in my experience, both in terms of promptness as well as rigor. The most common scenario with conventional journals is that most reviewers respond only after multiple reminders. It is obvious that many of them have not read the complete manuscript. They would raise questions which are elaborately answered already in the paper, would make sweeping statements without giving a single reference, or suggest citing references that are not relevant. In contrast, this paper received both appreciation and detailed critical comments. Differences of opinion were expressed without hesitation, but there were no derogatory remarks that I have seen so commonly in confidential peer reviews.
I spent a total 7-8 full days in reading, preparing detailed responses and revising twice, which I fully enjoyed. With one person I had an online discussion as well. Numbers are not enough to reveal the real fun of open peer review. The mindsets during the interactions were unique. In conventional peer reviews, while the authors are revising or replying to comments, scientifically sound arguments are not enough. They have to worry about satisfying the reviewer along with his preconceived notions, prejudices and egos. There is no choice but to please the reviewers. I have experienced this from both the ends and have written about it earlier (https://milindwatve.in/2020/09/17/how-peer-reviews-are-degrading-the-spirit-of-science/). This time I found myself only committed to logical soundness of arguments without worrying about the interpersonal complexities. Wherever we differed I had no hesitation saying that we differ on this issue. I had the same feel about the interacting reviewers too. This is great. This is a real platform for science.
This anecdote strengthens my notion that open peer reviews will improve the quality of reviews substantially. There has been a debate about open peer reviews multiple times on different platforms. Many people fear that this will increase the responsibility of reviewers and therefore they will be even more reluctant to review. We already have dearth of reviewers. Does 13 reviews and 2 revisions within one month support this fear? There were worries expressed that reviews will not be critical enough. Just read the reviews to this paper to see whether that is true, they are all there for everyone to read on the same link.
The most important point is that anything that is not transparent is NOT science. I have over 200 experiences of receiving peer reviews, dozens, if not hundreds of peer reviewing for journals until I started replying to review requests that I will accept only if you make it public. This is not a small sample size. I received high appreciation as well as irresponsible and insulting remarks, highly entertaining stupidity and “oh, how come I didn’t think of this” feeling. But still I would say this is the first time in my life I thought I was doing science and only science. On all previous occasions there was a feeling of either facing an exam or standing in the court of law. The word “peer” indicates standing on level grounds. First time in my life I was really standing on level grounds and talking only of science. In this sense this is the first “peer” review of my life. All others felt like being a criminal under trial. My sincere request to all fellow researchers to experience this, from both the ends, as authors and as reviewers. Open peer review is the future of science, if science has to be science all the time. Otherwise the flaws, biases, power play, inequality, racism, discrimination, regional imbalance, exploitation, bad incentives, reproducibility crisis, research misconduct which is evidently growing in the field will end up with the entire field harboring only pseudoscience.
I know that open peer review journals will have tough time for quite a few years owing to the religious and aristocratic social structure of the scientific community. Right now the community is deep in a pit of the illusion of journal prestige and CV building. The culture of evaluating one’s research without reading it has created a trap for everyone. Anyone with career worries will not be able to do open science for the fear that any attempt to do so will ruin my career. I must only try to publish in high impact journals. This cowardice will prevent science from being science for a long time, but I am an optimist. Science will certainly become true science some day.
My science, My way 