I have written many times about the problems in peer review systems. But most of this was from the author’s point of view. My experience from the other side is also quite rich. I have reviewed quite a few manuscripts for a variety of journals including some so called top ranking ones. This is not something I like to do, but it’s a part of researcher’s life, at least in the prevalent system.
A few months ago I received a manuscript for review. It was an interesting and quite off beat experiment. I liked the experimental design. The data and the analysis was all quite clean. But I thought the inferences that the authors had drawn were not quite logical. The results were being over-interpreted. The data only showed association but the authors confidently claimed a causal pathway without any further evidence. The results were specific to a given context but the inferences sounded as if a general law was being discovered. I wrote my comments accordingly saying that your experiments are ingenious but the interpretations need to be reconsidered. I also hinted that you don’t have to agree with my interpretation. If you think your interpretations are correct, you need to make them more convincing. A difference of opinion is not a sufficient ground to reject the manuscript, but kindly acknowledge in the paper that multiple interpretations are possible and reason out why you prefer one interpretation over the other.
The response of the authors was quite representative of a researchers’ behavior. Clearly they did not want to agree with my interpretation, which is fair. Having different opinion is a natural and desirable part of science. Open debates increase clarity, bring forwards many subtle aspects which would otherwise have remained hidden. But in a typical authors’ behavior with reviewers, they did not argue out their side. Instead, they pretended to agree with me, which obviously they did not. They did not want to change their original argument, but at the same time did not want to argue with the reviewer. So they rewrote the same inferences in a more round-about and ambiguous manner. I was irritated. I would have welcomed a clear argument, even if it were different from mine. Respecting the authors’ right to differ with the reviewer, and the experiments themselves being interesting I did not recommend rejection. Finally the paper was published. Being in a high impact journal, was read widely and as I expected, came under some heavy criticism.
Attracting criticism is not necessarily a bad sign. It is a part of healthy science. The part that was clearly against the spirit of science is that the authors avoided committing blasphemy against the ‘reviewer God’. I know that this is not a stand-alone incident. This is the modal behavior of authors, the reason for it being very obvious. I have taken up an argument against the reviewers at times ending up in rejection. In a typical response, the reviewers and editors are not in a mood to argue further. You simple get a sweetly worded rejection. Any logical debate is impossible. Most authors are smart enough to avoid that. Some are even ready to change their entire argument for getting acceptance. This is business and getting published in a high impact journal is such a huge benefit than you can easily trade the quality of science to reap the benefit.
A number of times we see papers where the data actually contradict the main claim made in the conclusions. I had the opportunity to ask one of the authors of such papers and he told me that the ambiguous looking conclusions were actually rewritten after getting the reviewer’s comments. There are two reasons for this trend. One is that a so called peer review is not really about a ‘peer’ relationship. It is more of a candidate and examiner relationship. An even better metaphor is that the editors and reviewers are God-men and the authors are worshipers. This makes practicing science one more religion where some are closer to the gods and others can access the gods only through them. The second reason is that the review remains confidential. So even if you make some logical somersaults there, it does not surface. What becomes public is a polished and painted argument covering up all logical cracks beneath it. The remedy for this is actually quite simple. Make all the reviews public, independent of acceptance or rejection. When everything becomes transparent, the arguments will have to become more logical. Differences of opinion will remain, but they will be open for the readers to judge. What can be better than this for the spirit of science? But do we really care for science?
My science, My way 
One thought on “How peer reviews are degrading the spirit of science:”