(Our response to Nature article “Predatory journals: no definition, no defence” https://www.nature.com/articles/d41586-019-03759-y)
Myself and Sonali Shinde wrote a reply to Nature (an abridged version of the following article) immediately after publication of the article. After two and half months they declined to publish it, on the grounds that they received many responses “making overlapping points” and that they will publish representative ones. Now let us wait and watch whether what we say below is represented in the set of responses they finally publish. I won’t be surprised if it is not because what we suggest here is what the science publishing community is deliberately avoiding to do for quite some time.
Intelligent Martians had been doing research on human behaviour for quite some time. Once a PhD student doing her observations through a superpower telescope, saw a mob of people doing something exciting. She called her mentor,
“Look, what these humans are doing there on earth.”
“Who are they? Men or women?”
“How would I know? They are not wearing any clothes!!”
This is precisely how a committee comprising 43 researchers, publishers, funders and policy makers from 10 countries that met in Ottawa, Canada last April is looking at the problem of predatory journals after two days of brain-storming. They want to identify predatory journals not by what they are but by what clothes they wear. The outcome of this meeting is published as an article in nature (Grudneiwicz et al 2019). The definition of predatory journals they have proposed is so subjective and open to interpretation that with a tighter strain, most mainstream journals can be labeled predatory and with slightly coarse one, none would be filtered out. The definition may not matter so much in practice but the diagnostic criteria would, since that is how one would identify a predatory journal in practice.
What should be the diagnostic criteria for a predatory journal? Charging the authors is a common practice now in so many flagship journals that it cannot be a differentiating criterion. Advertising in some form or the other is also practiced by some mainstream publishers and there is nothing unethical in it. The committee has listed certain other trivial criteria for diagnosis as a predatory journal. These criteria are either difficult to know before submitting (which the article itself acknowledges) or are so superficial that the journals can easily improve upon them and still remain predatory. For example, they list “an unprofessional looking webpage, spelling or grammar mistakes or irrelevant text” as markers of predatory journals. The act of identifying these as diagnostic markers would immediately make them improve on it, but that will not change the predatory nature of these journals. The biggest surprise decision of the committee is to leave out the quality of peer review as the defining or diagnostic marker. The only difference that can exist between mainstream journals and predatory journals is the quality of peer review. But they say “At the moment, journal quality, adequacy of peer review and deceit are too subjective to include……(as diagnostic criteria)”
This is precisely equivalent to identifying someone from the clothes and not from the being that he or she is.
It can be perceived without much difficulty that the main problem does not lie with the predatory journals, it lies with the mainstream journals. There isn’t sufficient transparency in the mainstream journals. The unnecessary confidentiality of the peer review process is the root cause of the problem. One of the reasons there is resistance to transparency is that the quality of peer reviews of the mainstream journals itself is often, if not always, questionable. Whenever an attempt to investigate has been made, biases and unprofessional behaviour of reviewers and editors of the mainstream journals has been found. This is well documented and published independently by several research groups (Campanario 1998; Bornmann et al 2010; Phillips 2011; Tomkins et al. 2017; Haffar et al. 2019; Kuehn 2017; Lee et al. 2013, Silbiger and Stubler 2019, Elson et al 2020). Even a subtle biases can permit persistence of a wrong paradigm and prevent acceptance of truth (Akerlof and Michaillat 2018). So no doubt is left that the peer reviews of mainstream journals are frequently of bad quality and are a major obstacle in the progress of science. Reviewers and editors can easily get away with bad quality reviews simply because they are never exposed. Systematic enquiries in the peer review process are also limited by the confidentiality (Couzin-Frankel 2013). It is most ridiculous that the main pillar of science, which is publishing, is not available for scientific inquiry. Making peer reviews public will certainly make editors and reviewers more responsible.
The threat of predatory journals is unlikely to disappear as long as the review process of mainstream journals remains confidential. Let there be dozens of committees like the Ottawa committee; let there be dozens of attempts to isolate and banish the so called predatory journals; the threat of predatory journals will not vanish as long as the mainstream journals do not themselves improve. If the presence of predatory journal really induces an introspection process in mainstream science publishing, I would say predatory journals is a boon to science.
But mainstream science is reluctant to take up the hard work. They think making superficial efforts like the Ottawa committee will work. They think they can diagnose and isolate predatory journals and then everything will be alright. This is not going to happen. In fact making lists of predatory journals is a dangerous solution since very soon it will become a business opportunity similar to the impact factor business. On the other hand, in the absence of clear and legally valid definition the agencies making such lists can be easily sued for defamation. So the entire attempt to list predatory journals and warn authors not to publish in them is a dubious affair.
The only long term solution to predatory journals is that mainstream journals make their editorial and review process completely transparent, independent of acceptance or rejection. This can be done using pre-print services and steps towards this goal are already underway (Brainard 2019). The unfortunate but not unexpected fact is that only 17 journals have subscribed to the scheme of open and transparent peer reviews so far. There are other means of making things transparent as well (Watve 2019). The reluctance to make things transparent makes one suspect that something fishy could be going on behind the curtain of confidentiality.
“If corruption is a disease, transparency is a central part of its treatment.” — Kofi Annan.
“A lack of transparency results in distrust and a deep sense of insecurity.” — Dalai Lama
If transparency becomes the norm of the peer review process, the entire reader community is free to judge the review quality. Then the so called predatory journals will either have to improve their review process, i.e. essentially cease to be predatory, or perish automatically. No formal committees and actions will be needed against them. Thus the definition of predatory journals is very simple- based on peer review quality and the only effective solution is also quite straightforward and that is transparency. But by evading a clear definition as well as the most logical solution, the scientific community is unnecessarily making the matters more complex.
Akerlof G. A. and Michaillat P. (2018) Persistence of false paradigms in low power sciences. PNAS, 115, 13228-33.
Brainard J. In bid to boost transparency bioRxiv begins posting peer reviews next to preprints. Science, https://www.sciencemag.org/news/2019/10/bid-boost-transparency-biorxiv-begins-posting-peer-reviews-next-preprints.
Bornmann L, Mutz R, Daniel H-D (2010) A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability
and Its Determinants. PLoS ONE 5(12): e14331. doi:10.1371/journal.pone.0014331
Campanario, J. M.: (1998) Peer Review for Journals as it Stands Today. part 1 and 2. In: Science Communication 19(3) pp. 181–211 and 19(4) pp. 277–306.
Couzin-Frankel J. (2013) Secretive and subjective, peer review proves resistant to study. Science, 341, 1331.
Elson M., Huff M. and Utz S. (2020) Metascience on peer review: testing the effects of a study’s originality and statistical significance in a field experiment. Adv. Methods Practices Psy. Sci. https://doi.org/10.1177/2515245919895419
Grudniewicz, A.et al. nature 576, 210-212 (2019). https://www.nature.com/articles/d41586-019-03759-y
Haffar, S., Bazerbachi, F., & Murad, M. H. (2019). Peer Review Bias: A Critical Review. Mayo Clinic Proceedings, 94(4), 670–676.
Kuehn, B. M. (2017). Rooting out bias. ELife, 6, 1–3.
Lee, C. J., Sugimoto, C. R., Zhang, G. & Cronin, B. J.Am.Soc. Info. Sci.Tech 64 (1), 2–17 (2013).
Phillips, J. S. (2011). Expert bias in peer review. Current Medical Research and Opinion, 27(12), 2229–2233.
Silbiger N. J. and Stubler A. D. Unprofessional peer reviews disproportionately harm underrepresented groups in STEM. PeerJ, 7, e8247.
Tomkins, A., Zhang, M., & Heavlin, W. D. (2017). Reviewer bias in single- versus double-blind peer review. Proceedings of the National Academy of Sciences, 114(48),
12708–12713. Watve M. G. (2019) The evolutionary psychology of scientific publishing: cost benefit analysis of different players in the game. https://ecoevorxiv.org/nvpe2/
My science, My way 