Especially for papers that rely on empirical work with painstakingly assembled datasets, the only way for peer reviewers to do the kind of thorough vetting that many commentators seem to imagine is implied by the words "peer review" would be to . . . well, go back and re-do the whole thing. Obviously, this is not what happens.
::Giggle::
(Sorry, that just slipped out.)
This is not to say that the peer review system is worthless. But it's limited. Peer review doesn't prove that a paper is right; it doesn't even prove that the paper is any good (and it may serve as a gatekeeper that shuts out good, correct papers that don't sit well with the field's current establishment for one reason or another). All it proves is that the paper has passed the most basic hurdles required to get published--that it be potentially interesting, and not obviously false. This may commend it to our attention--but not to our instant belief.
Fact-checking and the criticism of one's peers are not a few of McArdle's favorite things, and therefore a strawman is duly built, erected, knocked down, set on fire, and its ashes are sown into the dirt.
Welcome back, McArdle!
The peer reviews I've been a part of have always had certain criteria that put less weight on the conclusion of a piece of work than the data and analysis that it draws upon. This is done to deliberately protect unpopular ideas from being disregarded out of hand. Don't know if it's the same for all fields, but it would make sense if it were at least similar. So, once again, Megan is completely frigging wrong. But I wouldn't expect her to have much regard for data and logical analysis in the first place, so....
ReplyDeleteHer "revelation" reminds me of my mother indignantly explaining to me that she had recently discovered that Evolution [sic] was "JUST A THEORY"--something her teachers in school never admitted (insinuating that this was some academic conspiracy; assuming my mother actually paid attention in school and understood what was taught).
ReplyDeleteIntentional ignorance is a hallmark of our society, unfortunately.