Tag: ai

Who is more responsible? AI or scientists?

In my experience, which I am going to relate here, AI tools are sufficiently responsible while scientists are often not responsible at all.

The word “responsible” has a very clear meaning hidden in the word itself.  The ability and the tendency to respond appropriately whenever someone raises a question, challenge or objection to what you said/published. I tried doing this with AI responses as well as with research papers and scientific articles, editorials, reviews and correspondence by scientists. While AI tools always gave a serious response, scientists almost always (very rare exceptions) avoided giving any response. This was particularly remarkable when the questions raised were fundamental and challenged them in a serious logical and evidence-based manner.

See the responses of AI, in my earlier blog. I did not think what AI said was right. So, I cross questioned with reference to some public domain data and published papers. Then it accepted my arguments and made a change in its stance. On a few other occasions it said no…. notwithstanding my doubts the data shows so and so with reasonable certainty.  At times it said the issue is uncertain. All these are valid responses expected in the field of science. The response need not always be correct. It is not necessary that it “knows” the answer. For the spirit of science it has to be an honest response with the prevalent level of knowledge and always ready for rethinking and engaging in a dialogue or debate.

The response from scientists, in contrast, has been completely upside down. In the last couple of years, I posted serious comments/questions/challenges/alternative interpretations for about 20 papers and articles on various platforms, mainly PubPeer but also occasionally in Science e-letters and Qeios. Not a single author responded publicly. I wrote emails to individual researchers and the response was somewhat better. At least a few of them responded to emails, sometimes admitting that they were wrong, at times defending their arguments. Disagreement is a part of science and that kind of response is appreciated. But when I suggested it would be good to engage in a public debate, perhaps someone else may see something that both of us have failed to see, they were all reluctant. One of them said they would prefer having a debate on social media, but not in any science journal. In response I posted my challenge on social media, but still they did not respond. On at least two occasions I submitted a response to the journal as correspondence, in one case “matters arising” in Nature. But none of the editors entertained any debate. The rejection was without finding anything wrong with my arguments. On one occasion the editors asked us to pay 200 dollars just to consider sending our questions to the authors or considering them in any other form.

Often in the peer review process reviewers and editors make statements without giving any justification or references. Authors are supposed to do that, then why not editors? This is against the meaning of the word “peer”. On multiple occasions I asked for explanations to editors and reviewers about the statements they made in the rejection letter. I had not challenged the rejection, editors have every right to reject, I only demanded clarity on the statements that they made while rejecting. On this I received either no replies or the most irresponsible replies. Some of them I have made public earlier. I will make my correspondence with eLife, Lancet and and BMJ editors public soon. Some of them have given their consent to make the emails public but others I am still waiting. On one occasion the editor said we are rejecting because you are exposing too many facts!! We may consider publishing your opinion if you remove the original data. The funny replies showing that editors don’t even read emails properly (you ask something else and they answer something that you never asked) come from most prestigious journals. They would just not reply to any question that you asked. “I do not know”; “we never thought in this direction” or even “this question is not connected to the editors’ responsibility” are all perfectly valid answers, very much in the spirit of science. I would accept such answers. But instead, they would say something completely irrelevant to the question and end the matter there. One might argue that editors are too busy to answer your questions. This logically means that everyone is so busy in publishing papers that they have no time left to do any science. If prestigious journals are in the hands of such editors, how do we expect responsible publishing?

Is this only my experience? Perhaps I do not know the manners of the field and I may not have used the right language while raising the questions and challenges. I might be too arrogant for the field’s norms. But lack of responses is not restricted to my questions. Data show that only 7.5% of the comments on PubPeer receive any response from authors. That too is heavily dependent on who the person commenting is.

Holden Thorp, the editor of Science, had to write an editorial titled “Breaking the silence” in which he appealed authors to respond when anyone raises a challenge. This clearly means that failure to respond is extremely common in this field. The science editors also promised in this editorial that they will ensure that the authors of Science would respond. On this promise I wrote Holden Thorp giving examples where authors of Science articles had not responded, and asked him what action you consider on this as editor? To this question Holden Thorp has not responded till date, which is one full year.

So essentially, scientists do not even know the meaning of “responsibility”. How is it possible that they do good science? At least in this one respect, AI tools are better and more responsible than scientists. They respect your question. They may or may not agree with you. But at times they admit the mistake in their former response and correct themselves. At other times they may counterargue. All these are essential processes of science, without such debates and dialogues science can’t exist.

So as common man we need to cross question scientists and if they don’t respond, we need to question their right to be on those positions and drawing fat salaries from tax payers’ money.  There is no science without debate and dialogue and those who seem to run away from debates cannot be called scientists. They have no right to be there. Hopefully AI will take over and all such irresponsible scientists will be out of job soon !!

My diagnosis: Why Indian science remains mediocre

Science seems to be monopolized by a few elite institutions and a handful of prestigious journals. Science coming from all other places including India is mediocre (At least that is the prevalent belief.).  What is the reason for this? In my diagnosis it is certainly not infrastructure, funding, talent or collegiality. The true causes are the religious beliefs about journal prestige and peer reviews. The belief that only publishing in big journals makes science big has no foundation. The idea is a belief that has never been challenged or tested with data. It is there only because of the laziness to read. Reading the journal name is a lot easier than reading the paper and by human nature scientists only do whatever is easy.

The second belief, on the other hand, has been tested with data. The hypothesis that prestigious journals have more rigorous peer review system and they publish papers based on their quality has been tested with experiments and data and found absolutely wrong. Papers are accepted not by its content but by the prior reputation of authors and their institutions. This is very much evident by well-designed studies and analysis of editorial data. The behavioural origins of biases have also been made clear. So there is substantial literature on how much, how and why peer reviews are biased. But so far there was no consideration of the consequences of peer review bias. This I started doing in the form of a simulation model in this paper.

The logic is simple. Research funding, researcher’s motivation and publication output work in a positive feed back vicious cycle. Because of the auto-catalytic nature of the process a small peer review bias can result into a large difference in the output. And in reality, the peer review bias is not small by any standard. For the same manuscript, the odds of acceptance from a reputed location are over 6 times that of obscure authors. The difference between acceptance rates across countries is of the same order. Science published its editorial data analysis that shows a huge difference between acceptance rates of US versus China. India is not even considered in the analysis. As expected, much of the country based rejection is without reading and reviewing the content. They have admitted this difference and haven’t ruled out editorial bias as the main cause. They failed to reply to any cross question as well.

When a paper gets rejected, much time and efforts are needed to revise and resubmit, there are increased chances of it getting scooped during this time. Failure to publish affects further funding and thereby the downstream productivity. In simulations a small bias (10 %) in peer review resulted into one or two orders of magnitude difference in productivity even when no difference in research caliber was assumed. More important than that, the difference was escalated by increased competition. This is the reason why the problem is more intense in today’s highly competitive academia.

But more important is the second model of the paper which is about optimizing novelty. The model highlights the logical possibility that there an optimum level of novelty of research ideas which brings maximum success. Because of asymmetry in peer reviews the optimum for the elite and non-elite institutions can be widely different. In an obscure location, researchers doing mediocre work are more likely to be successful than researchers having out of the box, even revolutionary ideas. This is the most important behavioural reason why science coming from such places is mediocre. There may be and there is extraordinary talent in any and every part of the world. But extraordinary science is most unlikely to get published from obscure places. Talent gets discouraged very soon and by the innate tendency of the human mind to optimize, starts thinking mediocre, because that gives greater returns.

If this is true, the way out becomes clear. First researchers from non-elite locations need to stop giving any importance to getting published in prestigious journals. I have seen pulp or even fraudulent papers getting published very frequently in top ranking journals. So the contents of a paper need to be viewed independent of where it is published. The fact that peer reviews are inherently biased needs to be publicly admitted and publication policy changed accordingly. India is particularly lucky to have a set of journals published by many Indian academies which are free to authors as well as readers. The academies need to make the peer reviews open in the public domain independent of acceptance/rejection. Research evaluation, in my view is an inherently bad idea; but if inevitable, research published only in transparent peer review journals need to be considered. If this policy is declared, researchers will publish in open peer review journals. That would make peer reviews more responsible.

All this needs honesty and courage. If that itself is lacking (which, I am afraid, is true for typical Indian academics) then such people will only keep on doing mediocre science. The colonial dominance will continue using biased peer reviews as the main power weapon. Science will be monopolized even more and the global power imbalance will keep on increasing.

Since science is an economic and political power weapon today, peer review bias is subtly driving global economic, social and political imbalance and nobody seems to realize this. There is substantial literature showing strong peer review biases and imbalances, so why are we shy of even talking about it? I see cowardice as the only possible answer.  India has huge research talent but this cowardice will destroy India’s science. Any amount of funding and infrastructure support will not be able to change the picture qualitatively. We need to reject the prevalent publication systems and start from scratch on our own. I am sure, if such a bold step is taken it will revolutionize global science making it more equitable and humanitarian.