One more step in my commitment to transparency in peer reviews
The dictionary meaning of “peer” is “a person who is of the same age or position in society as you.”
This meaning of peer is expected in peer reviews. Peer reviewers are not like “examiners” of a student candidate. They are supposed to be at the same level of scientific standing as the authors. Therefore the norms and responsibilities of science writing that are expected from the authors should be applicable for the peer reviewers too. For example, author of a scientific paper is expected to support every important statement by either citing reference from prior studies or by his/her own logic and evidence. The same rule should apply to a peer reviewer. But rather too often it is seen that peer reviewers make sweeping statements without even a minimal attempt to support them. More alarming is the fact that editors have no hesitation in accepting it simply because it comes from a reviewer and not from the author. With such double standards this cannot be called “peer review” by definition. The minimum standards applicable to authors should be applicable to reviewers as well and editors do not have the right to give differential treatments to authors and reviewers.
But reviewers make irresponsible comments quite often and get away with it because peer reviews remain confidential. Editors cannot reject a review because reviewers have become a rare commodity. Hardly anyone is willing to “waste” their time in reviewing others’ work for which they get no credit. If editors insist on higher standards from reviewers, getting manuscripts reviewed will become orders of magnitude more difficult. Therefore they are compelled to compromise on the quality of reviews. As a result, the review quality is degrading rapidly and nobody cares because they remain confidential.
I am copying below the correspondence that I recently had with the journal “Evolution”. Let me state very clearly that Evolution is a very respectable journal and I have no intention to single out this journal for criticism. I completely respect the editors, some of them being good friends of mine. But all journals suffer the same ailment. I just happen to have a fresh example from this journal, therefore I am making this correspondence public.
I am sure that the editors won’t like this. I may convert my friends into not so much of friends. It is just too natural if this act affects my chances of getting papers accepted henceforth. I also consider the possibility that the journal takes any kind of action on me officially. But that’s ok. I am doing this as a part of my commitment to transparency of peer reviews. If only a handful of authors start making the reviews public on their own, the review process will become more responsible in no time. But any researcher who has to worry about his/her career will not be able to do this. I can afford to do this because I don’t have any career. So I can keep the broader interest of science above career concerns.
Earlier with the same journal, I received a review request and I wrote back that I am ready to review if the journal agrees to make the review transparent. They did not agree and therefore I declined to review. My commitment is therefore independent of my position. I did not raise the issue because my paper was rejected.
I am pasting the entire correspondence of both the incidences with the journal Evolution below for the readers to evaluate and interpret themselves. Improving the quality of peer reviews is just too important to keep it under the rug.
13-Feb-2023
Dear Dr. Watve:
Thank you for submitting your manuscript, “Evolution of new variants of SARS-COV-2 during the pandemic: mutation limited or selection limited?” (22-0518) to Evolution. It has been evaluated by Associate Editor Dr. Maria E. Orive and two reviewers, whose comments are appended below. Unfortunately, these evaluations, as well as my own appraisal, indicate that your manuscript is not suitable for publication in Evolution.
I am sorry that the review process took so long. Although I realize you will be disappointed by this decision, I hope that you find the feedback useful for considering submission elsewhere or for planned future directions with the work.
We appreciate your interest in publishing in Evolution and hope that this decision will not discourage you from future submissions.
Yours sincerely,
Dr. Miriam Zelditch
Editor, Evolution
zelditch@umich.edu
Associate Editor
Comments to the Author:
Reviewer 1 points out some very important issues with the approach taken by the analyses in this manuscript; the information known about the spread of variants of SARS-CoV-2 show that conditions given by the approach distinguishing between the hypotheses of selection versus mutation as the limiting factor for spread are not correct for such novel infectious diseases. As such, the approach taken appears to be fundamentally flawed.
Reviewer(s)’ Comments to Author:
Reviewer: 1
Comments to the Author
I do not think this paper is suitable for publication. Unless I have misunderstood something it puts forward a rather naïve approach for distinguishing between selection versus mutation being the limiting factor in the spread of variants of SARS-CoV-2, and it then uses this approach to infer from data that selection has been the primary limiting factor. This, despite the fact that we know this is probably not the case for all major variants that have spread, up to and including Omicron. Thus, to my reading the ms uses an approach that almost certainly can’t work for the data at hand to conclude something from the data that we know is almost certainly not true.
The problem with the approach is at least two-fold. First, the main focus of the approach assumes that immune-mediated selection is the primary selective factor but we know for novel host-pathogen associations like SARS-CoV-2 and humans, that the most important selective factor initially will be differences between humans and the ancestral host. When one incorporates that into a model one sees that the first several variants that spread will tend to be driven by strong selection for adaption to the new host, irrespective of immunity. So the conditions given for distinguishing between the hypotheses are not correct for such novel infectious diseases. And we know this is important in SARS-CoV-2 because Alpha and Delta (and probably Omicron) were all selectively advantageous regardless of the immune status of people in the populations. Second, the goal also seems to be to use the appearance of waves of infection in COVID as a potential signal of the underlying evolution but we know that this was not the case for many (perhaps most) waves for the first year or two. Instead, waves were almost entirely driven by changes in behavior mediated by public health measures and seasonality.
The problem with the conclusion of the ms is that we know for Alpha, Delta, and Omicron that selection was not the limiting factor. As mentioned above these variants were unconditionally advantageous, regardless of immune status. Further, the spread of these variants in a location was entirely migration limited. Each of them appeared in different geographic locations (England, India, South Africa) and then spread through migration to other countries. It was only once these variants arrived through migration that they increased locally, and they did so somewhat independently is different countries despite the countries having very different immunological histories. On top of that, each of these variants had a very unusual constellation of a large number of mutations, further arguing that mutational appearance was the most important limiting factor.
Incidentally, if one wanted to determine the role of selection versus mutation in the spread of variants then why not look at the phylogeographic data? If the process is mutation limited then won’t Delta variants across the globe, for example, have a common Delta ancestor? If instead selection is the limiting factor then, for example, Delta should spread at different times in different populations depending on their immune history, and they should not share a common Delta ancestor.
Other comments
P2 – migration should also be mentioned and included as an important evolutionary factor, particularly for SARS-CoV-2.
P3 – I think the characterization of models that incorporate immunity is unfair here. There is a very large literature on this. Furthermore, immunity can be binary and still display gradual loss of immunity at the population level (which is probably what matters here).
I also think that, throughout the ms, there is a tendency to conflate the issue of the spread of a variant with the occurrence of a wave of infection. These are distinctly different things. For example, in many locations, when Alpha was initially increasing in frequency, the overall number of infections was decreasing (which is why many authorities were hesitant to impose lockdowns initially).
Reviewer: 2
Comments to the Author
All the arguments made here are sound but they are not in the least novel. The authors have ignored 30 years of work in strain dynamics to start from scratch in understanding a very basic feature of the emergence of novel variants within a standard epidemiological framework which applies to SARS-CoV-2. They conclude, correctly, that the new waves are not likely to be “driven” by the emergence of variants – but, I’m afraid to say, that is a result that has been in the literature for at least 30 years.
14th Feb 2023
Dear Dr. Mirium,
Thanks for your letter of rejection and comments by two referees. Rejection is an inevitable part of the publication game and we take it in a positive spirit. Simultaneously we have some queries for which we would like to seek answers and clarity about the editorial policy of Evolution.
1. What is the meaning of “peer” by your norms? By dictionary meaning, peer is at the same level of social or organizational status. Therefore the set of norms for authors and peers should be comparable. If authors are expected to support every statement by citing appropriate reference or with new evidence, aren’t peers also expected to do the same? Both the reviewers of our paper make many sweeping statements neither citing relevant literature not giving any evidence. If authors’ manuscripts can be rejected because of inadequate evidence, do the editors have a policy to reject reviewer’s comments based on the same set of norms? How frequently in the last few years the editors of your journal have rejected reviews if they do not comply with the same norm as authors?
2. Reviewer 1 says multiple times that “We know that this is not the case…..” “We know that ….selection was not the limiting factor” and so on but does not indicate the data on the basis of which we “know”. In the absence of any supporting rigorous study cited, this can be at the best taken as the reviewer’s belief. We independently searched but did not find any rigorous study supporting the reviewer’s multiple beliefs. If it is a case of belief versus evidence, does your journal go by belief or by evidence? Let us clearly know the journal norms.
3. The two reviewers starkly contradict each other. Reviewer 1 says that our arguments are not sound and reviewer 2 says that they are sound but not new, that is, they are already well established. This clearly shows the irreproducibility of peer reviews. Does the journal think that in order to increase reproducibility of science, first there needs to be a minimum level of reproducibility in peer reviews?
Kindly understand that we are not challenging the rejection or appealing a reconsideration. We accept the discretion of the editor. We are only seeking some clarity about the journal’s general norms on which manuscripts are accepted or rejected. In the broader interest of science it is necessary to ensure that the norms follow the minimum necessary principles of science.
One more earnest request: We believe in transparency of peer reviews. Since the manuscript is on preprints archive, we would also like the peer review reports and our response to it respectfully posted on it. We would like to have your consent to post the peer reviews on the preprint server or any other appropriate public domain. It would also be in the interest of science that this correspondence is made public appropriately and respectfully. Transparency is the first requirement of science and I believe you will not disagree. So kindly let us have a written consent to make the entire editorial process for this manuscript public.
Thanking you and awaiting your response eagerly.
milind
| Miriam Zelditch | Tue, Feb 14, 9:03 PM | ||
| to Tracey, me | |||
I am forwarding your message to the Editor-in-Chief of Evolution.
—
Miriam Zelditch
| Tracey Chapman (BIO – Staff) | Feb 23, 2023, 5:24 PM (7 days ago) | ||
| to managingeditor@evolutionsociety.org, me, Miriam | |||
Decision on Manuscript ID 22-0518
Dear Milind,
Thank you for writing with your concerns and queries, which we have considered carefully. I also took a fresh look at your MS, the reviews and decision-making process. You raise three inter-related points: (i) assessment norms for authors vs reviewers; (ii) third party support for reviewer assertions; (iii) contrasting reviewer reports.
The reviewers of your MS gave different perspectives and contrasting reasons and assertions in their assessments. This is not unusual, and editors are trained to integrate sometimes disparate views, which, as in this case, may have varying levels of depth. They pick out the substantive, evidenced concerns, and down-weight others in order to come to a recommendation and then decision. Editors don’t ‘vote count’ but integrate the information according to professional standards https://academic.oup.com/pages/authoring/journals/preparing_your_manuscript/ethics?login=true#peer. We always aim to make it clear which are the key elements that feed into the final decision (though as reviews are supplied verbatim, that may not always be so apparent from an author’s perspective).
You are not challenging the decision on your MS, so I don’t comment in depth on any specifics. However, my assessment is that some substantive and legitimate concerns were raised by the first reviewer. For example, they described in some detail their concern, supported by observations, that the approach used could not distinguish between selection vs mutation. The expert AE comments specifically on this main point of concern to justify their decision and I agree with this assessment. The second review, written in somewhat stark terms I agree, was not central to the final decision (had the outcome had been different, we would have asked you to counter it). Overall, I find no concerns with the quality of decision-making rendered on your MS.
You also ask that we consent to make the entire editorial process for this manuscript public. This journal uses double-anonymised peer review, rather than a fully open peer review process, to minimise the influence of well-known unconscious biases on decision making (e.g. Ware et al. Info Serv & Use 28 (2008) 109–112). As part of this process, reviewers are informed that all communications regarding the manuscript are privileged, so sharing their reviews without their permission would raise potential ethics concerns. Therefore, we ask that you respect this confidentiality if you can.
Thank you again for raising your concerns, which we do appreciate,
Best wishes,
Tracey
Prof Tracey Chapman | School of Biological Sciences
Editor in Chief, Evolution
| Milind Watve <milindwatve@gmail.com> | Feb 24, 2023, 9:30 AM (6 days ago) | ||
| to Tracey, Miriam, managingeditor@evolutionsociety.org | |||
Thanks for your valuable response.
Since, as you agree and as so many studies unanimously demonstrate, the peer review process is inherently and seriously biased, any debate regarding the fundamentals of the process are most welcome and in fact badly needed.
I had asked three questions and you appear to have not replied to two of them.
I asked whether there are different standards for authors and reviewers by your journal norms. For example the authors are expected to support every statement they make, but reviewers are allowed to make sweeping statements without support. The answer to this question could only be yes or no. No other wording can be an appropriate answer. I also asked if there is a situation of belief versus evidence, what do your journal norms go by? I also did not find any answer to this.
Whether rejection to our paper is justified or not was not the question at all, and you have answered the unasked question quite elaborately.
I thank you for answering the third question that if the two referees contradict each other, the editors use their own discretion. In this you have also clarified what I asked. Editors do reject some of the reviews. The further natural question is whether the reviewers are conveyed your rejection as you do it for authors? If not, again do your journal norms permit different standards for authors and reviewers. It’s ok if the answer is “yes”. Only clarity and transparency in the norms is what I request.
I beg to differ with the last point. Double blind reviews are psychologically impossible. The moment a reviewer sees a manuscript his/her mind immediately starts guessing who it could be. This is human nature and can’t be surpassed. I have worked as a reviewer multiple times and asked so many other reviewers as well. In small fields it is frequently possible to guess. Further, preprint practice directly contradicts and nullifies the attempt to conceal the identity of authors. Since preprint is a well accepted practice, double blind peer review remains only a pretense. Transparent peer reviews is the only option that will improve the peer review process.
Since I am a strong promoter and supporter of transparent peer reviews, I am afraid I will make all attempts to make this correspondence public. I am a small man and nobody would notice it. But I have to remain honest to my principles. The journal is welcome to take any legal action against me if necessary.
I will also continue to communicate manuscripts to your journal, and if you decide to reject them on the grounds that I don’t agree with the confidentiality norms, kindly reject them clearly stating this reason.
My thousand apologies for all the trouble, but this is a necessary trouble in the broader interest of science.
Thanking you once again.
milind
(Dr. Milind Watve)
जुस्ते हक़ की रहगुजर में जो सियाही है, मेरी है
उस मजाज़े आराइश में तेरा ही बस हो तो क्या है
The darkness on the path to truth is my homeground. If the dazzling lights in the rest of the world are under your command, why should I care!!
Earlier correspondence related to me declining to review for the lack of transparency
| Evolution <onbehalfof@manuscriptcentral.com> | Thu, Dec 8, 2022, 11:10 AM | ||
| to milind.watve | |||
08-Dec-2022
Dear Dr. Watve:
Thank you for replying to my invitation to review for Evolution.
It is unfortunate that you are unable to review this manuscript at this time. I will keep you in mind when future manuscripts come in that fall under your area of expertise.
If you have a suggestion for an alternate reviewer and did not provide this on the site, please email the editorial office at managingeditor@evolutionsociety.org with your suggestion. Please also include the manuscript number EVO-22-0577.
Sincerely,
Dr. N. G. Prasad
Associate Editor, Evolution
Dear Prasad,
I wrote the following to the managing editor address.
Dear Editor,
I received a request to review a manuscript from your journal.
I am declining for the following reasons.
After being grossly disappointed with mainstream science, I have been working with undergraduate students, farmers, tribes and other people doing real and truely enjoyable science. While doing so I feel the hypocrisy of mainstream science even more. Thinking over a few years I have reached certain decisions.
I have been fighting for transparent peer reviews for quite a few years now. As an author, I am at the receiving end and can do little about the system. But whenever requested to be on the other side of the table, now I will demand a change in system. I will do reviewer or guest editorial jobs only if the journal is ready to make the entire process transparent. What I mean is that all reviews, comments and decisions are made public by appending them to the preprint on a public access server. This should be independent of acceptance rejection. With the opaque and therefore dubious peer review system, I think I should not waste my time because as it is peer reviews do no good to science.
I request your journal to rethink about the peer review system. If that happens I will give my 100% to it. If not, kindly do not request my inputs for any manuscript hereafter.
milind
(Dr. Milind Watve)
जुस्ते हक़ की रहगुजर में जो सियाही है, मेरी है
उस मजाज़े आराइश में तेरा ही बस हो तो क्या है
The darkness on the path to truth is my homeground. If the dazzling lights in the rest of the world are under your command, why should I care!!
My science, My way 