Review karma

Apr 03 2015 Published by under Academia, Decisions, publishing papers, science

When reviewing somebody else's paper for a journal, I found myself thinking:"hey, my paper just got rejected with a similar response to the reviewers! Perhaps I should reject this paper too!". Or I could think:"I want to submit a manuscript soon and would wish the reviewers would go easy on me, so I will accept this paper too."

How do you use your review-karma?

18 responses so far

  • drugmonkey says:

    This sort of thinking has to be rigorously eradicated. It is not about payback or back scratching circles.

    • babyattachmode says:

      It should be but then again isn't this how people do things?

      • drugmonkey says:

        Not the decent and good people, no.

      • Also Anon says:

        "It should be but then again isn't this how people do things?"

        Seriously, that's what you really think?! If so, then you are most definitely part of the problem! I'm totally with DM on this one. If you want good karma, do the right thing, period.

        • babyattachmode says:

          Of course I am as objective as I can as a reviewer. But it did get me thinking about how this kind of behavior changes requirements over time. If you keep getting presents from a friend that get larger and larger each year do you keep giving a gift for the same amount of money or do you increase too? And wouldn't it work like that with reviewing?

          • babyattachmode says:

            So to clarify: I don't mean this in a paper to paper situation, but I was thinking about this in the context of how requirements to publish change over the course of years.

  • newbie PI says:

    I recently invited one of the big shots from my field, i.e., one of my closest competitors, to give a seminar here. Knowing that I'm now one of the most likely reviewers for her lab's papers, she spent most of our 45 minute in-office meeting giving me a lecture on how to review papers. The jist was that I shouldn't expect perfection in every figure as long as there is really clearly something new and important that is well described. To be honest, I agreed with pretty much everything she said, but the clear implication was that if I'm nice to her papers, she'll be nice to ours. When I finally got to show her a few slides of our science, she stopped me and said, "make sure you recommend me as a reviewer for this. *WINK WINK* " It felt so smarmy, yet wouldn't I be shooting myself in the foot if I don't participate in this kind of back scratching?

    • Jim Woodgett says:

      Ugh. Do not go there. Firstly, there is no way you will know whether an editor will use a particular reviewer you recommend. Secondly, reviewers have ultimate deniability of anonymous. The more established scientist has cards already and will only keep them if you let him/her. Fields are bigger than you think and while a single reviewer can do damage, don't pander as it only perpetuates bad behaviour and entrenches the status quo.

      Young investigators are incredibly vulnerable but should never sell themselves short. You should go with your head, not heart and evaluate the science not the person. By the same token, young scientists must very careful about their anonymity. Be careful with unintended clues.

    • E rook says:

      I would not recommend her as a reviewer and I would COI my way out of reviewing her paper.

  • Jim Woodgett says:

    Reviewing is all about paying it forward. Rise above your own work and treatment and focus on:

    1. Whether the research is rigorous.

    2. Whether it oversteps its conclusions.

    3. Whether it lacks essential experimental information.

    Your recommendation is then simplified to whether it should be published, could be published or is unsuitable for publication. The tenor of review should be consistent with one of those outcomes and should never be influenced by your own situation, only your scientific experience.

  • […] Review as you would wish to be reviewed. […]

  • Kevin. says:

    Never assume the good and bad reviews come from your friends and enemies, respectively. Often, it's the opposite.

  • GMP says:

    Here's an additional aspect. Often, the answers to Jim's questions are yes, no, no. The problem is whether the paper is publishable in the particular (fairly to extremely) prestigious place. This is where things get murky, in my experience. I have had several papers rejected from what would be considered prestigious venues with essentially -- well, the paper is fine, but one or more referees don't think it's quite hot enough for this journal, so we will reject.

    When it's down to "The paper is fine and even interesting. But do I feel this paper is suitable for this fancy place in which everyone, including me, gets rejected more often than not?" I am afraid most humans end up being less than perfectly objective, because it's down to a subjective sense of "fit", of being worthy of publishing in a prestigious venue etc.

    • Jim Woodgett says:

      Suitability for the journal shouldn't be the reviewers concern. That's the editor's job. Reviewers job is the quality and rigor of the science (supposedly).

      • GMP says:

        Dunno... For the journals I review for, there are explicit questions as to how we (the reviewers) judge the suitability of the manuscript for the journal.

        • babyattachmode says:

          Yes! As a relatively n00b reviewer I struggle with this sometimes too: how much more do you expect for a fancier journal than for a less high IF journal?

          • DrugMonkey says:

            The best starting place is to compare it with topically-related work that has been published recently in that journal, of course.

Leave a Reply