Carrot and stick
(May 15th 2007) Peer review can be a painstaking process. Authors are on tenterhooks waiting for the evaluations of their papers; referees tend to hold-off on reading the papers. So the process can easily become a long - sometimes too long - drawn out haul. How can one give referees a hand? Blacklist the lazy or unreliable ones? Reimburse sedulity? Anja Possart reports.
Let's take a typical scenario in your life as a scientist. After months or even years of hard work, results finally make an interesting story to tell. You proudly put them into an adequate form, which is another serious and painstaking piece of work, and send the manuscript to a journal. If your story is interesting enough, the editors forward your manuscript to scientific colleagues and ask for a review. This period can be nerve-wracking because it may take months before you receive the comments - despite there being rules that referees should follow.
"Decisions will be made as rapidly as possible, and the journal strives to return reviewers' comments to authors within 4 weeks whenever possible", The Plant Cell promises on its website. Nature at least asks to be informed by the referee in case of delay and considers finding alternatives "where necessary". So there are rules - but rules, by definition, can be broken. Perhaps the referee is off hunting for funding, or he/she has more than just your paper on his desk, or there's lab work to do, or congresses are on the schedule... Referees are usually highly qualified researchers and, therefore, very busy. And sometimes it might just happen that your work is delayed because you are a competitor.
One way to make reviewers stick to the rules could be by introducing penalties for those who do not respect deadlines. A concept Marc Hauser (Harvard University) and Ernst Fehr (University of ZŸrich) recently considered in their article in PLoS Biology (April 2007, Volume 5, Issue 4). They suggest that journals should collect data on their referees' attitude towards deadlines and consider this along with the reviewer's next personal submission to the journal. Where lengthy delays have been recorded, the reviewer's own manuscript should sit in the editorial office for a certain amount of time, in a tit-for-tat action, before being sent out for review.
This is certainly an interesting way of attempting to properly channel the peer review process! It could be supplemented by a kind of blacklist that shows every delict, a list which could be exchanged among journals. Those appearing to carry out the job extremely slowly could then be deleted from the list of potential reviewers, at least for a certain period. Or perhaps journals could reintroduce a good old whack with the cane for those who don't behave properly.
Sounds good? Yes, but just a little too good to be true! Punishment would not be at all practicable. After all, the system of peer review is based on the referee's addiction to science, on fame and glory. You scratch my back and I'll scratch yours. Reviewing manuscripts is voluntary, even if it is a prestigious job from the scientist's point of view. Therefore, journals that introduce punishments would effectively stand to lose their best customers; the big shots of a scientific field would simply opt for a competitive journal instead of patiently allowing their own manuscript to sit on the back burner.
So, maybe one ought instead to be thinking about a system of reward for the right setting of priorities. Humans are highly sensitive not only to punishment but also to reward a fact well known by Hauser, a psychologist, and Fehr, an economist with a strong background in altruism and punishment, both being experts on human behaviour. They also suggest that counteractive to retaining the snail-referees' own work, journals should forward the manuscripts of "good" reviewers for review "as soon as they come in". Super! But once again, is that realistic? Would journals really prefer to push a manuscript which may not be a potential thriller, rather than a really fascinating one by the "bad" reviewer?
Well, humans, as we all know, are also slightly partial to financial gain. So what about paying for the referee's work? Maybe not directly and not for each reviewed manuscript but through a kind of account. If a referee consistently turns in a review on time, a credit could be agreed and this could be offset against the costs of the referee's next personal article. This would, at least, support younger scientists who might have problems paying hundreds of dollars for each colour picture. Big shots, of course, are well out of it again ...
There should be a solution to every problem. Do you have it? Let us know.
