I work in software, not academia. Would you mind giving me a quick overview of what referee reports are and how they apply? Most of the results on Google seem to assume you already know what they are, and the one concise answer I can find makes it seem like they’re job references?
Referee reports are reports produced – for free – by the peer review referees of their opinions / suggestions / comments on the papers that academic journals ask them to review on their behalf. Editors of the journals usually make their decision on whether a paper can be published as is, needs revision (and then to which degree), or rejected, according to the referee reports. In case of a revision, the authors of a paper will need to address the concerns in the referee reports in detail, often providing additional supporting materials etc. Referees are typically practicing scientists themselves and perform peer review for the good of the community (at least in theory). Most of the referees are anonymous to protect them from backlashes from criticizing the papers they review, particularly when some coauthor is a big shot. To be fair, not all referee reports are constructive, but at least the majority of them perform adequate gate-keeping to sift out the obvious bad apples.
Interesting. I’m not sure what the exact path forward would be, but a couple observations. The first is that I do think the lack of transparency into things like peer review hurts the scientific community overall. Often when I don’t fully understand why a piece of code is the way it is, reading the comments of the original PR can be really illuminating. Being able to see what sort of issues were raised about a paper during review and how the author handled them could, I imagine, add a lot of valuable context to the paper itself.
The next thing I’m noticing is something my academic friends complain about a lot: the vengeful, egotistic nature of many high-level academics. I don’t really know how to solve this problem when stuff isn’t anonymous, but I have to imagine that an author being able to see why their paper was recommended against and even being able to respond to it would be good for the author in the long run. In software you learn not to take someone deciding to reject your pull request personally, but also when someone rejects a good PR everyone else can also see that.
Overall I think the process helps blunt egos a little, and also shines a light on those that can’t get past their egos in a way that is also healthy for the community. But yeah, I hear you and don’t have a 100% good solution to egotistic retribution beyond vague noises about that being a cultural issue that needs to be changed and a vague sense I can’t back up with data that opening up the communication lines to public scrutiny might actually help shine a light on that culture and change it.
I work in software, not academia. Would you mind giving me a quick overview of what referee reports are and how they apply? Most of the results on Google seem to assume you already know what they are, and the one concise answer I can find makes it seem like they’re job references?
How do they relate to peer review?
Not sure I understand your question. What are you referring to by “that” and “it”?
Referee reports. I’ll edit to be more clear.
Referee reports are reports produced – for free – by the peer review referees of their opinions / suggestions / comments on the papers that academic journals ask them to review on their behalf. Editors of the journals usually make their decision on whether a paper can be published as is, needs revision (and then to which degree), or rejected, according to the referee reports. In case of a revision, the authors of a paper will need to address the concerns in the referee reports in detail, often providing additional supporting materials etc. Referees are typically practicing scientists themselves and perform peer review for the good of the community (at least in theory). Most of the referees are anonymous to protect them from backlashes from criticizing the papers they review, particularly when some coauthor is a big shot. To be fair, not all referee reports are constructive, but at least the majority of them perform adequate gate-keeping to sift out the obvious bad apples.
Interesting. I’m not sure what the exact path forward would be, but a couple observations. The first is that I do think the lack of transparency into things like peer review hurts the scientific community overall. Often when I don’t fully understand why a piece of code is the way it is, reading the comments of the original PR can be really illuminating. Being able to see what sort of issues were raised about a paper during review and how the author handled them could, I imagine, add a lot of valuable context to the paper itself.
The next thing I’m noticing is something my academic friends complain about a lot: the vengeful, egotistic nature of many high-level academics. I don’t really know how to solve this problem when stuff isn’t anonymous, but I have to imagine that an author being able to see why their paper was recommended against and even being able to respond to it would be good for the author in the long run. In software you learn not to take someone deciding to reject your pull request personally, but also when someone rejects a good PR everyone else can also see that.
Overall I think the process helps blunt egos a little, and also shines a light on those that can’t get past their egos in a way that is also healthy for the community. But yeah, I hear you and don’t have a 100% good solution to egotistic retribution beyond vague noises about that being a cultural issue that needs to be changed and a vague sense I can’t back up with data that opening up the communication lines to public scrutiny might actually help shine a light on that culture and change it.