Improving peer review

This is an interesting idea on improving peer review.

Real peer review has never been tried – Works in Progress

For example, in a large 2014 experiment at the Journal of Public Economics, a team led by the economist Raj Chetty tested four review processes: a control (with a six-week deadline for reviewers to submit their review), a ‘nudge’ of a shortened deadline (four weeks), a $100 cash reward (for submitting within four weeks), and a social incentive (where the times they took to review submissions were posted publicly).

For example, in a large 2014 experiment at the Journal of Public Economics, a team led by the economist Raj Chetty tested four review processes: a control (with a six-week deadline for reviewers to submit their review), a ‘nudge’ of a shortened deadline (four weeks), a $100 cash reward (for submitting within four weeks), and a social incentive (where the times they took to review submissions were posted publicly).

Cash incentives work BEST.

Here’s another:

On a centralized platform connected to different journals, scientists might be able to submit an article, be matched with reviewers, and select journals where they wanted to have it published – going through the process of submission and reviewing just once rather than over and over again.

Agreement between reviewers is even lower in the frontiers of a discipline, where the scientific consensus is less clear. But whether you believe this agreement is good or bad – perhaps because having a more diverse pool of peer reviewers who disagree with one another helps to prevent biases and possibly reconsider the consensus – there’s a problem. With only a few reviewers per paper, reviews are neither a sign of consensus nor intellectual diversity; rather, they are closer to a reflection of chance.

The author suggests a practical option:

Whether we like it or not, research is already, easily and increasingly, published outside of journals, and so are reviews. Reforming peer review, therefore, should mean working with the way science is shared in public, not ignoring it.

So the future is not just about using new tools and platforms for peer review; it’s about academia and industry investing in it wholesale by building institutions and teams to produce it and take the process forward. 

There are no easy ways from these suggestions. Part of the problem is that science publishing is “codeified”. If it is not broken, don’t fix it. However, journal editors have huge responsibility to take the conversation forward scientific progress, it often overlaps their desires for academic gatekeeping. I have had numerous papers rejected for submission, only to see something similar published at a later date. What changed? Owning the “weighty journals” is the best way to aggregate scientific ideas and profuse them in the practise. It can work specifically as a mathodology of exclusion and grant admission to cosy clubs and reinforce heirarchial structures. If it is done with the veneer of respectability, the blatant hypocrisy becomes even more apparent.

Most of us do realise that the “peer review” system is broken but only because it is used to whitewash the favoured submissions. I have seen enough duplication of “ideas”, yet they persist with the publication because everyone needs “content”. The effect of grant committes and the “big names” despite the masking, are apparent in the process. Besides, there are numerous “editorial services” to “re-write” the content to match the content guidelines submission process. There are whole publishing “firms” with “medical writers” dedicated to the process. How do you effectively compete with them?

Besides peer review, I am questioning the very specifics of academic publishing and raising questions on the veracity of the claims. Think about it.