Peer Review Is Fatally Broken

[Jccarlton]

Since the usual cry of "It's not peer reviewed" is going to come up I thought I would post msimon's take on it:
http://www.ecnmag.com/blogs/2013/10/pee ... lly-broken

[Schneibster]

Unbelievable.

You'll tell any lie, won't you?

You do know the anti-evolutionists claim that, right? And the YECs?

Peer review works fine. It keeps nutjobs like you on the lunatic fringe.

[Jccarlton]

A good example of the total breakdown of peer review:
A published paper from Mann and his friends and all it is whiny complaining because people are asking poor Michael Mann for stuff he supposed to supply:
http://wattsupwiththat.com/2013/11/01/m ... more-96600
Another common tool of harassment involves FOI requests. Under many legislations around the world, email correspondence by an academic is subject to almost unconditional release. During the last 9 months, the first author has been subject to numerous requests for correspondence and other documents, including trivial pedantry such as the precise time and date stamps of blog posts. In a paradoxical twist, accusations of impropriety were launched against the first author when an FOI-release confirmed that inconvenient research (Lewandowsky, Oberauer, & Gignac, 2013) was conducted with ethics approval. The allegations — by bloggers unaccountable to any form of review or ethical scrutiny — cited the fact that ethics approval was granted expeditiously (for details, see Lewandowsky, Cook, et al., 2013). The second author and his former university endured vexatious demands for the release of personal email correspondence by Virginia’s Attorney General. Those actions attracted national and international attention and were labeled a “witch hunt” by Nature (2010). The demands were ultimately rejected with prejudice by the Virginia Supreme Court. Other attacks on the second author involved front groups like the “American Tradition Institute” and the “Competitive Enterprise Institute” which sought access to his personal emails, professional notes, and virtually every imaginable document from his entire career. The third and fourth authors’ research center on tobacco control has been subject to a number of extensive FOI requests from a tobacco giant, Philip Morris International, for confidential interview records involving teenaged participants. Notably, the identity of Philip Morris was disguised during the first FOI request, which was launched with a law firm serving as a front group (Hastings, MacKintosh, & Bauld, 2011). The information requested included “all primary data,” “all questionnaires,” “all interviewers’ handbooks and/or instructions,” “all data files,” “all record descriptions,” and so on.
The problem is that when you work for the government the stuff you use in your work is part of the public record. I used to have to provide information when I worked for Jefferson Lab and my drawings were archived. You don't own your data, the public does and is entitled to view it. That's the price you pay for doing science for the government. In any case the fact that Mann and his friends were able to get this drivel through the peer review process and published demonstrates how broken the process is. This looks as if all the bad actors in science got together and put this thng out whine about how reality bites.

[Schneibster]

Peer review means review by a selection of scientists in the same field.

It's the ultimate in democracy, and you hate democracy, don't you?

[Jccarlton]

Spammers are broken souls.

[Schneibster]

Yep. And they deny peer review.

[Jccarlton]

spammers are the tools of bad science.

[Schneibster]

Then they don't do peer review, because peer review is what makes good science in the first place. Always has been.

[Jccarlton]

Spammers do bad peer review.

[Schneibster]

The only "bad peer review" lately was a magazine editor who published a climate crank paper without properly checking the peer review results. The climate cranks had lied, and the entire community laughed at him. He resigned.

Here's a report from Science, one of the two most prestigious scholarly journals in the world: http://news.sciencemag.org/2011/09/jour ... mate-paper

Here's another, from Physics World: http://physicsworld.com/cws/article/new ... ange-paper

[Jccarlton]

there is no point in peer reviewing spam. just dump it in the trash and move on.

[horsewithnonick]

Peer review means review by a selection of scientists in the same field.

It's the ultimate in democracy, and you hate democracy, don't you?

Peer review has nothing to do with democracy, as only a few individuals, selected by even fewer individuals, make judgments on the papers submitted.

It is supposed to detect and correct flawed methodologies and reasoning; there is a growing body of evidence that it is failing at this goal.

The alternative offered in the referenced post is at the very least thought-provoking, if one is into having one's thoughts provoked.

[Schneibster]

Peer review means review by a selection of scientists in the same field.

It's the ultimate in democracy, and you hate democracy, don't you?

Peer review has nothing to do with democracy, as only a few individuals, selected by even fewer individuals, make judgments on the papers submitted.

Apparently you forgot the editor who resigned over Roy Spencer's lies:

http://news.sciencemag.org/2011/09/jour ... mate-paper

Yes sir, it's democratic. Screw it up and you're gone. Get a new job, you'll never do science again.

It is supposed to detect and correct flawed methodologies and reasoning; there is a growing body of evidence that it is failing at this goal.

Unfortunately for your point it appears to have done just fine in the case of Roy Spencer.

The alternative offered in the referenced post is at the very least thought-provoking, if one is into having one's thoughts provoked.

Peer review invented the jet aircraft, the computer, the cell phone, and the refrigerator.

It seems to be doing just fine to me.

If you want my opinion, the climate cranks, physics cranks, and Darwin deniers outnumber the scientists and are trying to threaten them into shutting up, just like Giordano Bruno. You'd burn them alive if you could, wouldn't you?

[GIThruster]

Peer review has always had challenges. There's nothing broken about it. People with no credentials, like Simon, may want to pretend peer review is broken, but despite it is a plagued process, it is the best process we have available for understanding complex situations outside our field of expertise.

[Schneibster]

Peer review has always had challenges. There's nothing broken about it. People with no credentials, like Simon, may want to pretend peer review is broken, but despite it is a plagued process, it is the best process we have available for understanding complex situations outside our field of expertise.

I see nothing to disagree with there.

And in fact, Willard "Tony" Watts and National Review and the Competitive Enterprise Institute are about to lose a defamation lawsuit which will actually strengthen peer review enormously, in terms of reputation.