Jason, based on your posts, it appears you have an awful lot of experience dealing with the peer review process, in particular, the meteorology review process. I'm curious to hear your first hand accounts as to the atrocities that the peer review process has committed against you. If you don't have any first hand experience than I think you should be a little more careful with how unabashedly you attack something you have not experienced yourself. If you do have experience, then, again, I'm interested in knowing what has happened to shape your opinion in such a negative manner.
I can speak as someone who has published formally, has been rejected formally, is in the process of attempting to publish again, and a reviewer. Peer-review is a lot like democracy, it sucks, until you consider the alternatives. By no means am I a peer-review process apologist, but I do firmly believe that there must be some barrier to entry -- a bar to surpass, if you will -- regarding getting things accepted into the scientific literature. Note, this is the "scientific literature". Nothing prevents someone from holding on to their own beliefs, and sharing their results with others. However, if you want to get things accepted into the scientific record, scientists must have a say. Otherwise we end up rehashing the same "discoveries" and "inaccuracies" over and over again, making the new and really important discoveries harder to identify because the signal-to-noise ratio becomes extremely low.
Here's a hypothetical scenario. Let's say that there was a website aimed at the chasing community. Initially the website filled a small niche and thus the barrier to entry was a bit low, and only the most ardent of chasers were active on the website. After all, why would non-chasers want to spend their time discussing what light bar to put on their roof? However, as the popularity of chasing increased, the number of people visiting this website increased as well. Soon, the website was overrun with people who had no experience chasing and the forums were full of NWS product reposts, post asking where to go, and posts perpetuating meteorological inaccuracies. It wouldn't be long before people start demanding stricter membership requirements, such as writing a membership essay that is read by those in charge of membership to help restore the signal-to-noise ratio to earlier times. Is it the most ideal solution? Of course not, but it's a way to prevent self-proclaimed chasers who have never chased from becoming labeled as a "chasing expert". How is that different from scientists reviewing the work of other scientists to protect the integrity of all scientists? I find it ironic that some of the most ardent opponents of the peer review process are also those who demand a stricter requirement for membership on Stormtrack.
No one (at least in meteorology that I'm aware of) requires a meteorology degree when publishing. If the non-degreed scientists want to publish their research, they are more than welcome to. The catch is, they are held to the same standards as degreed scientists. If, after scrutiny, then their work holds up, it should pass through the review process. Occasionally bad things do get published and good things get withheld. And then there are the cases like my rejection where a reviewer takes umbrage with a previous paper and decides to take it out on the paper before him. We all have our horror stories; we all think it can be better.
As is the case in most small scientific fields, or which meteorology is one of them, it isn't too hard to figure out who reviews your submissions. It behooves reviewers to be professional in their reviews of colleagues, because there is a good chance that he or she will be reviewing one of your publications. This is why in most scientific publications, reviewers must justify their reviews to an editor. It is then the editor's decision to accept or reject the paper -- not the reviewer's. Furthermore, most articles are reviewed by a minimum of 2 scientists, and most often it is at least 3. Again, is this the best scenario? No. However, it's better than just letting anything be accepted.
Let me also clear up another thing. As of earlier this year, Reed's PhD dissertation was not related to tornadoes nor any of the data he might be collecting. In fact, unless it has been recently changed, it has nothing to do with severe weather (at least in the United States).
