The journal eLife has declared its new peer review policy which is a bold experiment in science communication. In a recent editorial (https://elifesciences.org/articles/83889) they declare that eLife will not use peer reviews for a dichotomous decision of accept-reject. Instead they will publish every paper that they choose to review along with the reviewers’ comments. Further they say that the authors will have sufficient freedom to use this peer review to publish elsewhere etc.
To some extent, this is precisely what I had been saying for quite some time (but with important differences). Of course, many others would have thought that way. As the article says, “Nobody who interacts with the current publishing system thinks it works well, and we all recognize that the way we use it impedes scientific progress”. Since there appears to be a consensus that the current system of science publishing is deeply flawed, there need to be alternative models and there have been some experiments on alternative publishing models. But one thing is lacking.
Science is a human endeavor and therefore is clearly subject to principles of human behavior. Any new system being designed needs to be based on our understanding of behavior. If it is based only on ideology, but ignores behavior, it is bound to fail in realizing its objectives. It may become stable and popular but that is not the measure of success. How far it serves the original purpose should be the measure of success. The central question is how to design a system that will serve the purpose, given the behavioral choices of all stakeholders in the field. The reason why the existing system is flawed lies in people’s behavior and a new system can also get easily corrupt for the same reason. It is therefore necessary to analyze the reasons behind the problems in the current system and see whether we have been addressing these problems in designing a new system. I have written a detailed article which is available as a preprint for the last 5 years (https://ecoevorxiv.org/nvpe2/). As expected in the article itself, this couldn’t have been published by a traditional journal, and the prediction has turned out to be correct so far. In these 5 years my analysis of the behavior of scientific community has gone much ahead. Here I will only mention a couple of behaviorally important points relevant to the eLife experiment that has begun.
The committees that decide recruitments, promotions or funding look at where a candidate has published rather than what is published. This is not without reason. The journal names and impact factors save them the cost of reading. Reading incurs substantial cost. IFs are popular only because they save the cost of reading. There can be an inexpensive pretense of evaluation without evaluating anything. So although IFs are not scientifically sound, they are behaviorally profitable and therefore the committees will not give up on them easily. The eLife’s stand of replacing the accept reject-decision by publishing peer reviews will compel the committee members to read research, and they will be most reluctant to do so. For over 2-3 decades, committee members are addicted to the ‘evaluate without reading’ package and de-addiction is not going to be easy.
The accept-reject decision cannot be replaced as long as the prestige of journal matters. The more prestigious journals will be overburdened with submissions and they can review only a limited number. So desk rejection will become even more important and there all the biases caused by the dichotomous decision will return in perhaps a worst form. eLife itself says “We will publish every paper that we send out for review”, which means a large number will be rejected without giving reason at some one’s vim. This decision is bound to be guided by private cost benefits of the editor which is not going to eliminate the existing biases.
There is one more potential contradiction. The elites of science control most of the prestigious journals. Therefore they are not so unhappy about conventional peer review systems. Peer reviews have biases by gender, country, race, reputation etc. So the underprivileged class of science will find the open peer review system more beneficial. But mostly the underprivileged are also poorly funded. They will not be able to afford the author charges of 2000 dollars per paper. So the change may not benefit the ones who are looking for a change. The journal has a facility of waiving charges, but how efficiently it works will decide everything. It is quite likely that the profitability or even sustainability of the journal will be compromised if waivers are really given to everyone who needs. There are more problems with the suggested change. But nevertheless, any experiment is welcome. The risk in doing such experiments without sufficient thinking is that failure of such experiments will further strengthen the flawed system once again. Therefore it is necessary to design behavior based systems right away. In economics and management, designing behavior informed systems is not a new concept. There is substantial research on it. Why not utilize this in science? And if the field of science itself fails to use novel scientific concepts, who else will?
My science, My way 