Comment
Are we training pit bulls to review our manuscripts?
Journal of Biology 2009, 8:24 doi:10.1186/jbiol125
- The author reflects on comments received
- The solution for 'manuscript-savaging reviewers'
- A paper's own merits
- Never savagery for savagery's sake.
The solution for 'manuscript-savaging reviewers'
Colin Anderson
(2009-03-16 18:17) University of Alberta 
The article by Virginia Walbot is an excellent primer for reviewers. However, it is surprising that the simplest solution to the "manuscript-savaging reviewer" problem was not considered. The solution is to encourage or even demand that reviewer’s identities are not hidden. Non-anonymous reviewers are much less likely to make spurious claims or demands. Instead, due to the accountability that comes with open reviewing, the reviewers restrict their criticisms to those they are quite certain of. The usual argument against open peer review (e.g. as practiced by the journal Biology Direct) suggests that the critiques will not be sufficiently rigorous. This should not be seen as a strong argument against open review, as the reviewer’s reputation is in fact much more on the line in opening reviewing. There will be much more pressure to catch the fatal flaws and ignore the subjective gut responses, responses that are all too frequent in rapidly done anonymous reviews. Given the time demands we all face, I would surmise that the true origin of resistance to open review lies in reviewers not wanting to have to take the time that the accountability of open review demands. The primer for reviewers (and editors) should at least include an encouragement to sign reviews.
Competing interests
None
A paper's own merits
Mark Gijzen
(2009-03-16 18:16) Agriculture Canada
There definitely seems to be a rise in excessively critical and disingenuous reviews. But there is also poor handling of manuscripts by editors, who often show bias or misunderstanding of the subject. My feeling is that much of this arises from our current over-obsession with journal prestige and impact factor. Perhaps open access models will help to put more emphasis on judging papers based on their own merits, rather than the title of the periodical it is published in.
Competing interests
None declared
Never savagery for savagery's sake.
Ian Burgess
(2009-03-16 18:16) Medical Entomology Centre, Insect Research & Development Limited
One of the first things I learned as a reviewer is that you should never write something
that will not benefit the authors either to improve the manuscript or as a lesson
for future writing. I agree that with the introduction of online reviewing it is
easy to jump in with documenting flaws without first reading the whole manuscript.
However, I also feel that with online submission some authors are sending in their
work more hastily than they would once have done had they needed to forward a pile
of paper to the editorial office.
Apart from training our students to identify the strengths and weaknesses of others'
papers we also need to train them to look critically at everything they write. Many
is the manuscript I have seen that clearly has not been reviewed internally either
by the author's peers or supervisor or, if it had been, the products of that review
show either that the internal reviewers were too close to the trees to see the forest
or else they have skimmed over the weaknesses, presumably hoping nobody else would
see them. Maybe some of them are too timid to point out the flaws in the work of
their supervisees, some are not perceptive enough to identify the issues, and others
may fear that a little criticism is not what is required. However, better a mild
analytical criticism at home where the issues can be discussed face to face than to
be savaged by some unknown using a few short sentences to try to convey points that
really need extended discussion. Since a high proportion of manuscripts give acknowledgement
to departmental seniors, and even outsiders, for their "helpful" comments on the manuscript
I frequently wonder what it must have looked like before it was let out.
Too often I come across vague suggestions as to why a set of results are not as expected
but the authors invariably appear to have made no attempt to determine whether their
suggestions may be correct, or even state that this line of enquiry is being pursued
as a separate investigation. If the results being put forward for publication are
preliminary this needs to be said. If the money or time ran out before all the angles
could be addressed, why not say so? If you've run out of ideas you should not fear
saying "I don't know how to approach this" rather than giving a lot of flim-flam that
will only attract vitriol. I know there is considerable pressure to publish rapidly
from all sides, but a little quiet reflection about how other people might view what
you have written should surely reduce the number of poor manuscripts entering the
publications arena.
I really do like reading other people's research. It gives me great pleasure to be
able to write that a project was soundly conceived, the results discussed roundly,
the conclusions are valid, and a manuscript is well written. Sadly it is something
I encounter less and less frequently, so training students to analyse past publications
may be working in some fields but doesn't seem to be doing so in mine.
Competing interests
No competing interests
The author reflects on comments received
Virginia Walbot (2009-03-17 10:30) Stanford University
It has been rewarding to receive gracious e-mails and posted comments about how to better train young scientists as thoughtful reviewers. Of course, I also heard from pit bull owners in defense of their pets. I stand by the analogy, however, because a good owner trains the dog to be a safe companion, while a bad owner can incite vicious and dangerous behavior – and the pit bull’s jaws are strong enough to sever your arm! Furthermore, it’s easier to train than retrain a dog, or a person. This is exactly the point of training graduate students and postdocs: we need to use class time for this excerise and also explicitly discuss our own reviewing habits with our lab groups to reap future benefits of thorough and reasonable criticism.
Several commentators also pointed out related essays that interested parties may wish to consult: http://www.healingtherapies.info/Objective%20Science.htm and http://jcs.biologists.org/cgi/reprint/113/24/4373
Finally, I think it’s important to evaluate how the recent changes in the Journal of Biology reviewing policies play out over time. Can journals mitigate the angst felt by authors by tweaking the reviewing system? Will reviewers become more thoughtful if they receive fewer “rewrites” of nearly ready to publish manuscripts and can hence concentrate much of their review on the list of required alterations that authors can readily achieve without subsequent outside reviewer approval? Should we maintain the cloak of anonymity in our current system? These are all issues ripe for the community comment.
Competing interests
Author of the original article.
top