What does peer reviewed mean? how do you get something peer reviewed, and what are their specific standards?

Question

What does peer reviewed mean? how do you get something peer reviewed, and what are their specific standards?

Answer 1

Here's an explanation of the process, as used by the International Journal of Climate Change

http://ijc.cgpublisher.com/peer_referee_...

It's definitely not a case of reading your friends paper and patting each other on the back.  Only someone that's never read a science journal or been involved with any scientific research would think that.

Answer 2

Research papers are submitted to the editor(s) of a scholarly journal.  The editor(s) ask two or more scientists who have research experience in the subject area of the paper to write anonymous critiques of the paper.  The reviewer analyses the experimental methods used and may ask for clarification of the methods or even recommend that experiments be repeated if flaws are found.  Next the reviewer looks at the data to see if it looks reproducible.  The reviewer looks at other publications to check whether the data is consistent with prior results.  If not, the reviewer anonymously asks the author(s) of the paper, throught the editor, to explain why the data is different.  Often the data is different due to improvements in the experimental design, but the reviewer must verify this.  Often the reviewer will perform a few experiments in his/her own laboratory to check for reperoducibility (I do).  Next the reviewer checks the results and discussion to ensure that proper procedures are followed in data analysis.  The reviewer will often point out a factor that was not considered and may request further analysis.  Last, the reviewer evaluates whether the paper will be of interest to other researchers.  The critiques are sent to the authors who then answer all of the questions raised and send a revised version of the paper back for review.  In practice, the critique is constructive and better papers than originally written are finally published.  I might add that as a reviewer I have recommended papers for publication because I think the data should be in the public domain whether I agree with the interpretaion of the data given by the authors or not.

Edit: pegminer (above) has a good answer.

Answer 3

Some great answers in the mix, and a great question, thanks.  I have a few points I thought I'd toss out there to flesh out the topic.

From the first link below:

"Peer review means new scientific discoveries, ideas, and implications are not accepted or considered valid until they have been scrutinized, critiqued, and favorably reviewed by other scientists who are experts in the same area or scientific field."

And on standards:

"The peer-review process sets a scientific standard; we know that peer-reviewed scientific work has been subjected to rigorous scientific evaluation by experts in the appropriate field and has been judged valid."

And finally, from the 3rd link:

"The system of `peer review’ was established during the nineteenth century as a means to uphold quality control in science and to exclude patently flawed science from the publications of the scientific community, known as `journals'. This of course involves something of a trade-off between the wider social values of free speech and the narrower values of preserving the integrity of science itself.

The ideas and papers excluded from the journals in this way could always be published in non-scientific publications so that the censorship only really applies within the journal community. Since the emergence of the internet, it is now possible for anyone to publish material without editorial interference - something seen as a dangerous curse by some, and an opportunity for genuine free flow of ideas by others."

So that's what we have here.  The uncontrolled "free flow of ideas."  Keep the salt handy.

Answer 4

It used to mean that uninvolved third parties would review the report to test the validity of the hypothesis. But, as we have seen with many of the AGW report, it means having friends or associated review the reports. This is especially obvious with many of the IPCC reports. A good example is the 3rd reports famous hockey stick. It was published after being reviewed by people with a direct interest in its publishing. Of course, once it was published, the entire theory was disproved. Of course, had it really been peer reviewed, it would never have been published to begin with.

Answer 5

Peer review is supposed to be like your father or teacher looking at your homework before you pass it in to check for errors.  This was done after several magazine articles proved to be questionable, the publishers wanted a second set of eyes to look at the work before the magazine went to print.

The concept went a little too far when it was assumed that peer review was a high standard that proved the article was correct and accurate.

However like minded people reviewing an article is not that high of a standard.  Since a person already believes what is written is true, what's to question or check?

Imagine a Scientologist writing about Scientology and letting other Scientologists review the article before it was printed in a magazine.  What do you think would be the conclusion of the peers?  Would you think that they would say Scientology wasn't real?  I don't think so.

Answer 6

Peer review (also known as refereeing) is the process of subjecting an author's scholarly work, research or ideas to the scrutiny of others who are experts in the same field. Peer review requires a community of experts in a given (and often narrowly defined) field, who are qualified and able to perform impartial review. Impartial review, especially of work in less narrowly defined or inter-disciplinary fields may be difficult to accomplish, and the significance (good or bad) of an idea may never be widely appreciated among its contemporaries. Although generally considered essential to academic quality, peer review has been criticized as ineffective, slow, and misunderstood.

Pragmatically, peer review refers to the work done during the screening of submitted manuscripts and funding applications. This normative process encourages authors to meet the accepted standards of their discipline and prevents the dissemination of unwarranted claims, unacceptable interpretations and personal views. Publications that have not undergone peer review are likely to be regarded with suspicion by scholars and professionals.

Answer 7

peer review usually refers to being published in a peer reviewed journal.

you submit you article (usually about your research) to a professional journal, like the Lancet. Journal of IEEE,

Journal of th APA, etc.

the standards of the various journal vary, but almost all of them want to see statistical analysis, and will reject you for sloppy statistics, interpreting your statistical results sloppily, or overstating the generalizeablity of your results.

Answer 8

Peer reviewed means that a paper you have written is copied and sent to people you may or may not know, but who are supposed to review it anonymously and recommend to the editor whether or not it should be published and what if anything should be corrected before publication.  Anyone that thinks it consists of your buddies sitting around and approving your paper has never been through the process.  It can be very frustrating and infuriating, but usually the paper ends up the better for it.  Depending on how thorough reviewers are they can go over every little nitpicky thing in your paper or they can skim it and pass it on.  Since there are usually two or three of them though, it usually gets pretty good scrutiny from at least one of them.

Answer 9

Viper353, d/dx+d/d and pegminer are correct, quite a few here claim to be scientists, from some of the subtle things in their comments i'm pretty sure d/dx+d/d and dana probably are.

I don't claim to be a scientist but I do work with a group of scientists who study Antarctica and the upper atmosphere one of the things I do is prepare diagrams for papers that are going to be published. The process of submitting is as Viper353, d/dx+d/d and pegminer stated it is a process that usually takes months and several sets of rewrites, referee's are experts in the field and are chosen because of lengthy periods of well received work in the particular field.

Dr Jello's comment here as usual shows his lack of knowledge of even a basic understanding of science.

Journals like JASTP, Nature, Science, JAS & GRL have nothing to do with nonsense about questionable magazine articles if jello actually did research instead of just try for the highest points he would know that the peer review process has been in wide use in science since the 1950s and few scientists take seriously any journal that isn't peer reviewed.

Answer 10

If you are really curious what peer-review looks and sounds like, go here:

http://www.atmos-chem-phys-discuss.net/p...

Atmospheric Chemistry and Physics is published by the European Geophysical Union.  It has a relatively high impact factor on SciCitation Index, and in the discussion section you can read the actual peer-reviews and author responses.  If you search, you can even find Lindzen's Iris hypothesis discussed at length (the refutation paper of the Iris Hypothesis was published in ACP a few years ago).  The only thing you can't see, sometimes, is the thought process of the editor as he/she evaluates a split review and decides whether a paper should be accepted or rejected.

Answer 11

Peer reviewed means the journal will have a team of their experts examine your paper before the journal publishes it.  The process is anonymous, so the experts can speak freely without worrying about offending anyone.

Who the experts are, and the specific standards they use, are the choice of the journal.  The standards are published by the journal so that everyone knows what they are.

Long experience has shown that the scientific merit of peer reviewed articles is considerably greater than that of articles not peer reviewed.

Answer 12

Peer review process is quite controversial.  Depending on the process it could improve the quality of work or it could be simply the triumph of politics over the scientific method.  

In the field of AGW research, it can be used to completely exclude studies that have repeatable scientific basis and lend some legitimacy to studies which are simply not repeatable.

The existence of Micheal Mann's Hockey Stick is evidence that the peer review process isn't improving the quality of science.  Nobody can reproduce his findings using his raw data.

Answer 13

When like minded people agree.

There are no standards, other than those self imposed by the like minded people.

For example, a group of your friends review something you write.  Then your writing has been peer reviewed.