Over a decade ago, I met a highly successful US based scientist of Indian origin. He routinely published, and publishes even now, dozens of papers in the most prestigious flagship journals. I have read much of his work with interest and have seen that he has consistently delivered excellent publications on the cutting edge of science. I asked him whether he thought he could do the same quality of research if he worked in India. Without having to think for a moment, he answered frankly, “At least in my field, I have no doubt, I can do the same quality work in India. What I am not so sure is that I would be able to publish it the same way!” He added further, “Publication is a mafia. If I am not in the circle, I won’t be able to publish. It is not enough to have good quality work, you need to have connections in the influential circles.”
Several years later I had a conversation with a celebrated scientist, a Noble Laureate himself. We were together for three days and he listened to my work and my ideas with interest. Then he said, “With a mind like yours, I won’t be surprised if you publish in Nature, Science, Lancet, PNAS quite frequently.” My spontaneous response was, “Try publishing from India and you will know.” He did not blink, only nodded. He did not need more than a moment to imagine and agree fully and instantaneously.
Everyone seems to know how the world of scientific publishing works. I won’t use the word ‘mafia’ myself. It is better left to discussions over coffee with successful elite scientists. But like every researcher I know that scientific publishing is a field loaded with so many weird problems, only a few of them surface once in a while and receive some attention, discussion and debate. Being a student of behaviour, I see many fascinating behavioural phenomena happening here, which make wonderful research problems. But the existing system of scientific publication does not permit any research of this kind. This is mainly due to the sacred confidentiality of the editorial process. The field of meta-science or the science of science is an extremely weak, almost non-existent effort. The handful of low ranking researchers in this field do publish something realistic and insightful once in a while, but which largely remains ignored by the scientific community and therefore has hardly made any impact on main stream science so far. Although ultimately it aught to make an impact, at present they are constrained by unavailability of data, which is largely due to the unnecessary confidentiality of the editorial process.
Most working scientists are fully aware of the flaws in the peer review system but they consider them ‘necessary evil’. Most seem to think that there is no better alternative and therefore let the system be what it is. But what if the evil is growing exponentially? Perhaps it is, and that is being felt as well. One cannot show the trend using any data, because all data remain hidden. There is nothing more ironic in the field of science. Entire science is based on availability of data and the main pillars of science are interested in hiding all data. Today science is moving rapidly towards a culture of open access, open source; where most journals now insist that authors make their raw data public in some form or the other. But the same journals are not ready to make the review process transparent and accessible to the public.
What really goes on under the carpet of confidentiality is left to anybody’s imagination. We only need to know that scientists are humans too and everything that goes on in the human world, does happen in the world of science too. Just that it remains hidden. Some journals now publish the reviewers’ comments along with the paper, but this is limited to accepted papers alone. All rejection related correspondence always remains hidden and there lies the real problem.
Interestingly, unlike the popular belief among students, a confidential review process is not a time tested tradition. It is fairly recent in the history of science. Out of the 300 papers published by Einstein, only one was subject to anonymous peer review and the comments were quite negative. Einstein himself made some nasty comments on it and withdrew his manuscript never to publish again in any journal with confidential peer review. Nature considered peer reviews as mandatory only by 1967 and Lancet by 1976. Mandatory peer review exists only for one or two generations of researchers. This is not long enough to say that it is a time tested system. The system has remained stable only because all possible ways of challenging it are blocked. Unless journals make the review data and particularly rejection data available to any meta-science researcher, the system cannot be claimed to be fair. Whenever, the peer review system has been tested, it is found to be highly flawed. There are only a handful of attempts to test it and they are published as well (for example see 1-6). So the most important activity in science, that of publishing stands on demonstrated unscientific principles.
The confidentiality of peer review has given rise to an entirely different problem, that of the so called “predatory journals”. Predatory journal is a big trap created by editorial confidentiality of the mainstream scientists. At present there are problems even in defining a predatory journal. There is no way to demonstrate that these journals publish without good quality peer reviews. They can always escape saying that we follow rigorous peer reviews but they are confidential so we cannot disclose. There is no use blaming such journals because this ghost is created by the mainstream journals themselves.
Can scientific publishing be made scientific? Can it be brought out from its religious fervour? We can certainly answer this positively and the way to do so is also very clear and simple. Bringing transparency in the editorial correspondence is the only and ”simple” way out. The reason I put the word simple within quotes is that it is conceptually simple, but not simple to execute because it will change the power structure in the field and so the currently powerful people will be reluctant to give up.
There is some justification for confidentiality and some of it’s reasons are genuine, but not intractable. The review process can be made public without affecting the anonymity of reviewers, or keeping anonymity optional for them. Many journals already have this option and this takes care of almost all potential problems arising out of transparency.
It’s not that things are not changing. Many journals are exploring different options for peer review. Some tried double blind peer review. What could be a sound looking idea, is actually not behaviourally sound. If a reviewer receives a manuscript from unknown authors, by human nature, the first reaction is to guess who could the authors be. The focus of the reviewer then goes on picking up indirect cues by which the author identity could be guessed. This guesswork does not help in eliminating biases, it might make it even more weird. So double blinding cannot be said to be a successful alternative. But there are more practical alternatives and there are attempts to try them out.
Last month we had a pleasant surprise. On communicating a manuscript to PLOS Biology, we got a request from the editors for our consent for an open public appeal on BiorXivs and on twitter for comments on our manuscript. We gladly consented. There were hardly any comments in the public domain, apart from brief ones showing positive interest, but based on three comments confidentially received by the journal the editors rejected our MS. I wrote back saying in the spirit of open peer review that we have consented for, the comments and our responses to them should be posted along with the pre-print. After a series of emails, two of the three reviewers and the editors consented and we posted the comments and our responses on BiorXiv (see the comments section of https://www.biorxiv.org/content/10.1101/553016v1 ). I think making the peer review public is a significant historically important development. Now BiorXiv accepts posting reviewer comments and author responses for a rejected manuscript. They declined to disclose the name of the journal. But the journal itself had tweeted that they are reviewing our manuscript, therefore BiorXiv’s stand of not disclosing the journal name is meaningless anyway. I feel the rejection was a blessing in disguise and a first hand experience of this new tradition is a greater reward.
On this background, I have made one more decision. I will also make the comments received on my other manuscripts public on my own blog along with our responses to it whenever appropriate. I see no reason for not disclosing the journal or the editor’s name. Reviewers mostly remain anonymous anyway. This activity will be independent of acceptance or rejection. I will also write back to the editor asking them to inform the reviewers that we have published your comments, and our responses to it, if any, and you are welcome to react on it further. I am sure that although a small step, this will potentially be a turning point. If a number of authors start making the reviews public, the reviewers will have to be more responsible for the comments they write. This increased sense of responsibility, by itself can reduce the problem substantially.
Is it illegal to do so? I see no reason. On at least half a dozen times, after getting a rejection, I asked the editors would they mind if I make all the editorial correspondence public? On this I received an amazing diversity of responses, including sheer panic, desperate defence of editor’s rights to rejection (which I had not challenged in any case). I further asked many of them whether they would take a legal action if I make them public, and under which law? For this they did not know whether there was any legal protection to the confidentiality of the editorial process. But without worrying about legality, I have decided to do two things in this blog. One is that I will publish my analysis of optimization of behavioural strategies of the different players in the scientific publishing game in a series of articles. This is a fairly large volume of work that could potentially make a thesis. But I will restrict myself to a series of (non-peer-reviewed) articles. The other is that I will start publishing the comments that we received for manuscripts communicated from our lab, one by one on this blog. This will apply to previous years too. I will encourage other researchers, particularly from the younger generation to watch this and decide whether they too would like to make the reviews public. If a critical mass (which need not be too large) of researchers start doing so, at the best it will bring in a revolution in scientific publishing in no time. At the least, it would provide excellent research material for researchers in history and philosophy of science and meta-science.
See these links to the editorial correspondence and peer reviews (along with follow up correspondence in some cases) of rejected manuscripts from our lab during the last few years.
- Rejection by Diabetes Care for our manuscript “How much variance in insulin resistance is explained by obesity?”. After receiving the rejection letter, I asked the editor, whether I can quote some comments from the rejection letter in any future article. I received a very long response. In this letter the editor actually agreed completely with our main argument. The long letter did not say anything about not allowing me to quote. So I am publishing the entire correspondence Here.
- We wrote a response letter to an article published in the Lancet, which the editors declined to publish. Then we wrote to the authors directly that for us publishing was not important but we had raised a question and would like to know your response to the question, which may be in the form of a personal email to us. The authors did not respond to this.
- A network model of type 2 diabetes, that had somewhat surprising and non-conformist outcomes, which was ultimately published by PLOS ONE, was rejected first by a number of leading journals. In none of the rejections, out central arguments were challenged. The reasons for rejection are very interesting and worth reading. Through subsequent submissions, the manuscript did improve a bit, but the comments received were hardly of any use towards improvement. You will find the editorial correspondence with Cell, Cell Metabolism, PLoS Biology, PLoS Computational Biology, Biology Direct and PLoS One here.
I will keep on publishing more editorial correspondence about rejections as well as acceptance one by one on this blog.
- Phillips J. S. 2011 Expert bias in peer review. Curr. Med. Res.
Opin. 27, 2229–2233. - Lee et al 2013 Bias in peer review. J. Am. Soc. Ino. Sci. Tech. 64, 2–17.
- Haffar et al 2019 Peer review bias: a critical review. Myo Clinic Proc. 94, 670-676.
- Tomkins et al 2017 Reviewer bias in single versus double blind peer review. Proc. Natl. Acad. Sci. USA. 114, 12708-12713.
- Kuehn B. M. 2017 Peer review: rooting out bias. eLife 2017;6:e32014
- Tancock C. 2018 When reviewing goes wrong. The ugly side of peer review. Elsevier Connect March 23, 2018.
My science, My way 
Looved reading this thank you
LikeLike