A short post to express a bit of anguish and to vent. I apologize to my gentle readers in advance. The subject of the peer review system has been discussed elsewhere in greater detail (a notable example is an excellent post by Jonathan Eisen, blogger and Professor at UC Davis). I believe in it, because I firmly believe that serious, conscientious peer review can promote the cause of science and scientific progress.
A while back, I had received an invitation to review a manuscript for a fairly well-circulated journal, and agreed to do it, as the subject matter was well within my area of expertise. Upon reading the manuscript, I found that there were several lacunae in it, in terms of methodology, data interpretation and conclusion. I, however, didn’t recommend outright rejection, because there was scientific merit in the conception of the work, and I thought it necessary to have the information – of the kind embodied in the paper – out in the scientific literature. That, and also because I am painfully aware of the urgencies associated with publishing one’s work. I went through the manuscript assiduously, checking references, eventually pointing out line by line, page by page, where the problems lay, and suggesting how the authors could improve it. I sent it back with the recommendation to ‘Modify’.
It appears that the Editor for the article agreed with me, and returned a decision of ‘Modify’ to the authors. I received a notification to that effect. Initially, I had a quick case of warm fuzzies, because I thought I was able to help the authors; with some modification, their work could be published.
And then I saw something, scrolling down, which completely deflated me. The notification contained all the comments from all the reviewers. Following my detailed, eleven-paragraph response (as reviewer 1) to the authors, I found the responses from reviewer 2 and reviewer 3 – they wrote precisely ONE paragraph each, giving a very general and vague summary of the study, without making any recommendation as to the acceptability of the paper.
Several questions quickly coursed through my mind:
- Is this the true face of peer review – one single hastily scribbled paragraph, determining the value of painstaking work by researchers spending time and money and effort?
- Was I being unfair on the other reviewers, who may be so much more experienced than I am, that one look at the manuscript and they were able to visually separate the grain from the chaff?
- Was I – a mere postdoc who didn’t know any better – an idiot, an irredeemable non compos mentis, to have spent time and effort and care to review this paper in a constructive way?
- Would I want my own papers to be evaluated in this way?
Of course, like many other puzzling mysteries of life, these questions, too, leave me clueless.