Reading Science: Peer Review – Part 4 in a Series

Peer Review

By nature journals are very conservative, and work very hard to exclude incompetent, poorly carried out or documented, or fraudulent science. They also work to evaluate the methods used in papers submitted for consideration, all this is the process of peer review. They critically, or are supposed to, read any conclusions and discussion in the paper, to ensure they are justified and supported by the theorem proposed, and by the methods used, and observations made.

These efforts mean that unless you really find them interesting, reading the methods used, or the discussion of statistics associated with many papers, is not necessary for our purposes. Those portions of the paper exist for the peers, actual scientists. Whose job it is to evaluate the research, and to endeavour to reproduce or to disprove it’s conclusions. Either using the same methods, or by adopting alternative tests if they feel they are better.

That is not my purpose, I doubt it is yours.

Although the efforts of the journals via peer review, etc., to screen for fraud and incompetence has had varying degrees of success, for our purposes, as with almost all scientists, we can accept papers as competently written and researched if published in a peer reviewed journal. Until they are proven otherwise, so you can safely ignore those portions of any academic paper concerned with methods.

But do not take that as my endorsement of these journals, of peer review or of the infallibility of their methods. There have been numerous examples where fraudulent science has gone undetected for years, and in recent times some examples where the gatekeepers of a variety of journals have been tested and found sorely wanting.

It is of course possible, common even, to get published by paying a fee. In fact it is central to the process, no paper gets published without someone paying for the privilege.

It goes far beyond the potentially corrupting influence of pay to publish, in a field where publishing is how one is judged. To get a glimpse into how screwed up the whole process is you can read this, and this. That will be the subject of a different series of posts another time.

Unfortunately this conservatism can make journals resistant and exclusionary, to anything radical or new – to them, beyond that required by scientific rigour. Science, even medical research, does not have as an explicit objective making people well. I have a hard time imagining a patent clerk in Zurich today being able to publish anything as radical as Einstein’s General Theory of Relativity was in 1905. Provenance matters too much to journals, who you are rather than what you think and how you express it come into play to our detriment. They also, in a very unscientific way, prejudge papers and exclude those they deem as unimportant. Which is a little cart before the horse. Of course it does protect against wishful thinking, people so desperately want to be well they are prone to see results or good when none exists.

But Helminthic Therapy suffers from these and other inbuilt prejudices and restrictions. It is low risk, it is not a drug in the sense that helminths are a novel molecule which could and likely will have unexpected and unwanted consequences. But relative risk is not something anyone takes into account. Because it is next to impossible to patent any aspect of the therapy big drug companies, as noted elsewhere, are not inclined to invest in the area. Drug companies, the big ones, depend on the artificial monopolies created using patents to extract monopoly profits from new drugs, as witnessed by their many and often immoral attempts to artificially extend patents using the legal system, legislation, repackaging and reformulation, etc.

In part because of this alternative journals, enabled by the internet, have been founded. These are often dedicated to areas of enquiry neglected by the mainstream journals, or that allow science produced, often necessarily, under less rigorous circumstances than possible for papers published by older journals. They sometimes adopt modern ideas like open source in the peer review process, by publishing unedited texts or opening the peer review process to comment and participation.

Examples of such journals in the area of medicine are Plos One,

Many journals focussed on medicine began as in-house brochures or magazines published by drug companies who wanted to market their latest patent cures to doctors. In the case of Bayer a notable drug they invented and marketed in this manner was Heroin which they promoted as a less addictive alternative to morphine. Merck followed a few decades later in one of their “journals” promoting Cocaine for, amongst other things, its power to treat Heroin addiction.

These were not isolated examples, in the past many papers on drugs were written by doctors or scientists hired by drug companies, which often contained little we would recognise as science. They were primarily concerned with promoting the virtues of specific drugs, not with scientific niceties. The FDA and it’s ilk did not exist until relatively recently, and there were no requirements for studies into safety or efficacy for new drugs. If you had one you could sell it, and promote it in any way you wanted.

Those papers were published without any of the safeguards that we take for granted now, like peer review. Things are different now, to a degree, but human nature has not changed. Witness the tobacco industry’s ability to find legions of scientists willing to put their names to papers for years arguing there was no evidence that tobacco was addictive, or that it was carcinogenic. If someone is paying then someone else is willing to put their name to almost anything.

Bear this in mind when reading papers, but do not fall into the trap of the conspiracy theorists. Most scientists are sincere and well intentioned, as I continue to hope most people in all walks of life are. Besides that do you really think large groups of anyone could keep their mouths shut?

But the story of journals while interesting is outside the scope of this article.

Once a scientific paper has been written up, usually involving multiple authors and many reviews within the group writing it, and often involving informal peer review amongst colleagues, it is submitted for publication.

There it is read first by a single reviewer to determine if it is worthy for further consideration, for peer review. Over half of submitted papers are rejected at this stage, usually to be submitted to less prestigious journals.

The reason for rejection, in effect a kind of censorship a priori, can include the following and more. It can be because the reviewer does not believe the science is worth reading, or that it lacks relevance. That it proves or elucidates nothing new to the canon (to the existing knowledge in that area). It can be rejected because it does not fit the editorial profile of the journal to which it has been submitted, drug research is not going to fly in a Physics journal for instance. It can tossed because it is prima facie poor science, uses poor methodology, draws unsupported conclusions. Or it could be rejected because the reviewer, conceivably, does not understand it and therefore thinks it absurd. Perhaps because they dislike one of the authors. Papers are often submitted to more than one journal, though doing so reduces the odds of inclusion in the more prestigious ones, exclusivity is a big deal when publishing research. If it is any good you want your journal to be the publisher, not one of a hundred.

Once accepted by a journal for further evaluation, publication is still not assured at this point, it is subjected to a process called peer review and may be withdrawn by the author/s.

Peer review is exactly as it’s name suggests. The editors send it out to other scholars in the same field (the author’s peers) to get their opinion on the quality of the scholarship, its relevance to the field, its appropriateness for the journal, analysis of statistical methods, fact checking, checking of the math used, etc. Peer review is not generally a paid activity. Though publication almost invariably is. That is once peer review is complete, perhaps before, the authors or their sponsors must pay the journal for publication, and this cost is often many thousands of USD, sometimes in excess of ten thousand dollars.

If it passes the process of review, often with numerous changes and even reruns of experiments, or the addition of other data not originally included but gathered, and the fees are paid it is queued for publication.

Having been published the article, along with letters to the journal about recently published papers, and follow up papers or confirming studies concerning previous papers and their results, as well as any studies commissioned by the journal, goes online.

From there it is indexed and catalogued by various research indices and search engines, like Google Scholar or Pubmed.

Where science is published and how to find it – Part 3 in a Series

To understand how to find and consume science, particularly on the topic of medicine, one has to understand how it is produced and published, and some of its past.

Science has not always existed, nor has medicine as we know it. In the relatively short time that the ideas of the Enlightenment have prevailed in some areas of our life, and science and medicine have been practiced, it has changed enormously.

Continue reading “Where science is published and how to find it – Part 3 in a Series”

What is Science? – Part 2 of a Series

Before we start reading scientific papers we should all agree what science is. We all know what science is, right?

So what is it?

From the Oxford English Dictionary (Shorter)

“Theoretical perception of a truth, as contrasted with moral conviction (conscience).”

Sounds a little loose a definition for my purposes, and like it is a derivation of the argument between rationalists and theologians at the beginning of the Enlightenment. Perhaps I should have bought the full OED.

Continue reading “What is Science? – Part 2 of a Series”

Reading research for non-scientists – Part 1 of a Series

Overview

I taught myself how to read science, even going so far as to dive into statistics so I could understand what “p” meant. I did so originally so that I could understand the hygiene hypothesis, old friends hypothesis, and what came to be known as helminthic therapy. Later I continued to read it so I could do a better job helping clients, but primarily because I had grown to enjoy it.

Reading scientific papers is one of my favourite activities. I even have a “greatest hits” list of my favourite papers, which I reread. I have learned an enormous amount from the activity, and derived even more pleasure. Because of that I wanted to encourage others to do likewise.

Continue reading “Reading research for non-scientists – Part 1 of a Series”

New direction for this blog

I have not had an editorial position for this blog, until now. No consistent direction or unifying theme for what to say except in general terms to speak about helminthic therapy and anything that might, however distantly, relate to the health of those who approach us for hookworm, or for whipworm.

Often I have been embarrassingly guilty of writing self-indulgent garbage I should have known was of interest only to me. Things that in retrospect should not have been of interest to me.  I apologise, it won’t happen again.

I have decided that I am going to concentrate on science for a while, what it is, how it is practiced, how it is funded, who decides what gets funded, what is published and how, and perhaps how elements of that might be improved. Science has not always been like this, or even been at all.

Continue reading “New direction for this blog”