Fabulous slide show, showing the problems with peer-review in CS today

If you are interested in peer review, see this presentation by Jeffrey Naughton.

Slides 14-20, 28-30 and 35 onward are particularly relevant.

A key point is that over-tough reviewing is often caused by reviewers themselves wanting to treat others just like they have been treated. There needs to be a way to break this cycle.

Some ideas in the presentation:

• All reviewers in a conference should be required to propose accepting a certain percentage of papers
• Reviewing where the reviewer can't see the author's name, but the author can see the reviewer
• No reviewing at all - let the market determine the best papers based on links and citations

However there is still a lot of 'bad science' that gets submitted to conferences. This could be solved by having reviews published along with each paper. Authors would want to avoid bad comments, and would also be able to respond to them and update their paper. See my earlier post on liquid publication.