Reading Science: Peer Review – Part 4 in a Series

Peer Review

By nature journals are very conservative, and work very hard to exclude incompetent, poorly carried out or documented, or fraudulent science. They also work to evaluate the methods used in papers submitted for consideration, all this is the process of peer review. They critically, or are supposed to, read any conclusions and discussion in the paper, to ensure they are justified and supported by the theorem proposed, and by the methods used, and observations made.

These efforts mean that unless you really find them interesting, reading the methods used, or the discussion of statistics associated with many papers, is not necessary for our purposes. Those portions of the paper exist for the peers, actual scientists. Whose job it is to evaluate the research, and to endeavour to reproduce or to disprove it’s conclusions. Either using the same methods, or by adopting alternative tests if they feel they are better.

That is not my purpose, I doubt it is yours.

Although the efforts of the journals via peer review, etc., to screen for fraud and incompetence has had varying degrees of success, for our purposes, as with almost all scientists, we can accept papers as competently written and researched if published in a peer reviewed journal. Until they are proven otherwise, so you can safely ignore those portions of any academic paper concerned with methods.

But do not take that as my endorsement of these journals, of peer review or of the infallibility of their methods. There have been numerous examples where fraudulent science has gone undetected for years, and in recent times some examples where the gatekeepers of a variety of journals have been tested and found sorely wanting.

It is of course possible, common even, to get published by paying a fee. In fact it is central to the process, no paper gets published without someone paying for the privilege.

It goes far beyond the potentially corrupting influence of pay to publish, in a field where publishing is how one is judged. To get a glimpse into how screwed up the whole process is you can read this, and this. That will be the subject of a different series of posts another time.

Unfortunately this conservatism can make journals resistant and exclusionary, to anything radical or new – to them, beyond that required by scientific rigour. Science, even medical research, does not have as an explicit objective making people well. I have a hard time imagining a patent clerk in Zurich today being able to publish anything as radical as Einstein’s General Theory of Relativity was in 1905. Provenance matters too much to journals, who you are rather than what you think and how you express it come into play to our detriment. They also, in a very unscientific way, prejudge papers and exclude those they deem as unimportant. Which is a little cart before the horse. Of course it does protect against wishful thinking, people so desperately want to be well they are prone to see results or good when none exists.

But Helminthic Therapy suffers from these and other inbuilt prejudices and restrictions. It is low risk, it is not a drug in the sense that helminths are a novel molecule which could and likely will have unexpected and unwanted consequences. But relative risk is not something anyone takes into account. Because it is next to impossible to patent any aspect of the therapy big drug companies, as noted elsewhere, are not inclined to invest in the area. Drug companies, the big ones, depend on the artificial monopolies created using patents to extract monopoly profits from new drugs, as witnessed by their many and often immoral attempts to artificially extend patents using the legal system, legislation, repackaging and reformulation, etc.

In part because of this alternative journals, enabled by the internet, have been founded. These are often dedicated to areas of enquiry neglected by the mainstream journals, or that allow science produced, often necessarily, under less rigorous circumstances than possible for papers published by older journals. They sometimes adopt modern ideas like open source in the peer review process, by publishing unedited texts or opening the peer review process to comment and participation.

Examples of such journals in the area of medicine are Plos One,

Many journals focussed on medicine began as in-house brochures or magazines published by drug companies who wanted to market their latest patent cures to doctors. In the case of Bayer a notable drug they invented and marketed in this manner was Heroin which they promoted as a less addictive alternative to morphine. Merck followed a few decades later in one of their “journals” promoting Cocaine for, amongst other things, its power to treat Heroin addiction.

These were not isolated examples, in the past many papers on drugs were written by doctors or scientists hired by drug companies, which often contained little we would recognise as science. They were primarily concerned with promoting the virtues of specific drugs, not with scientific niceties. The FDA and it’s ilk did not exist until relatively recently, and there were no requirements for studies into safety or efficacy for new drugs. If you had one you could sell it, and promote it in any way you wanted.

Those papers were published without any of the safeguards that we take for granted now, like peer review. Things are different now, to a degree, but human nature has not changed. Witness the tobacco industry’s ability to find legions of scientists willing to put their names to papers for years arguing there was no evidence that tobacco was addictive, or that it was carcinogenic. If someone is paying then someone else is willing to put their name to almost anything.

Bear this in mind when reading papers, but do not fall into the trap of the conspiracy theorists. Most scientists are sincere and well intentioned, as I continue to hope most people in all walks of life are. Besides that do you really think large groups of anyone could keep their mouths shut?

But the story of journals while interesting is outside the scope of this article.

Once a scientific paper has been written up, usually involving multiple authors and many reviews within the group writing it, and often involving informal peer review amongst colleagues, it is submitted for publication.

There it is read first by a single reviewer to determine if it is worthy for further consideration, for peer review. Over half of submitted papers are rejected at this stage, usually to be submitted to less prestigious journals.

The reason for rejection, in effect a kind of censorship a priori, can include the following and more. It can be because the reviewer does not believe the science is worth reading, or that it lacks relevance. That it proves or elucidates nothing new to the canon (to the existing knowledge in that area). It can be rejected because it does not fit the editorial profile of the journal to which it has been submitted, drug research is not going to fly in a Physics journal for instance. It can tossed because it is prima facie poor science, uses poor methodology, draws unsupported conclusions. Or it could be rejected because the reviewer, conceivably, does not understand it and therefore thinks it absurd. Perhaps because they dislike one of the authors. Papers are often submitted to more than one journal, though doing so reduces the odds of inclusion in the more prestigious ones, exclusivity is a big deal when publishing research. If it is any good you want your journal to be the publisher, not one of a hundred.

Once accepted by a journal for further evaluation, publication is still not assured at this point, it is subjected to a process called peer review and may be withdrawn by the author/s.

Peer review is exactly as it’s name suggests. The editors send it out to other scholars in the same field (the author’s peers) to get their opinion on the quality of the scholarship, its relevance to the field, its appropriateness for the journal, analysis of statistical methods, fact checking, checking of the math used, etc. Peer review is not generally a paid activity. Though publication almost invariably is. That is once peer review is complete, perhaps before, the authors or their sponsors must pay the journal for publication, and this cost is often many thousands of USD, sometimes in excess of ten thousand dollars.

If it passes the process of review, often with numerous changes and even reruns of experiments, or the addition of other data not originally included but gathered, and the fees are paid it is queued for publication.

Having been published the article, along with letters to the journal about recently published papers, and follow up papers or confirming studies concerning previous papers and their results, as well as any studies commissioned by the journal, goes online.

From there it is indexed and catalogued by various research indices and search engines, like Google Scholar or Pubmed.

Where science is published and how to find it – Part 3 in a Series

To understand how to find and consume science, particularly on the topic of medicine, one has to understand how it is produced and published, and some of its past.

Science has not always existed, nor has medicine as we know it. In the relatively short time that the ideas of the Enlightenment have prevailed in some areas of our life, and science and medicine have been practiced, it has changed enormously.

Continue reading “Where science is published and how to find it – Part 3 in a Series”

What is Science? – Part 2 of a Series

Before we start reading scientific papers we should all agree what science is. We all know what science is, right?

So what is it?

From the Oxford English Dictionary (Shorter)

“Theoretical perception of a truth, as contrasted with moral conviction (conscience).”

Sounds a little loose a definition for my purposes, and like it is a derivation of the argument between rationalists and theologians at the beginning of the Enlightenment. Perhaps I should have bought the full OED.

Continue reading “What is Science? – Part 2 of a Series”

What is corruption?

When I still lived in the US and Sani Abacha was the suddenly the dead ex-president of Nigeria, and it was therefore discovered he had looted billions from the Nigerian treasury, some of my friends in the US were aghast at the scale of his corruption. Apparently they did not read much international news, or much news at all perhaps.

My comment was “At least their leaders get a decent price”,  being of the view that to sell your constituent’s interests out to lobbyists in exchange for providing you the means to retain your position, as what, at the next election so you can repeat the process lacks ambition, any understanding of pricing, as well as integrity. That seems to me to be an accurate description of the relationship between lobbyists and the politicians they fund in America.

As I say, at least Mr. Abacha got a decent price for his integrity.

Which perhaps partially explains this map. One of world corruption based on the research and perspectives of those who prepared it. As the title asks, what is corruption?, so must have the map makers. How corruption is defined and measured, which varies with time and place of course, and who does the measuring. These fundamental things have an enormous impact on the results in such an exercise.

I think they missed most of the endemic corruption woven into western economies, of the type above and below. Corruption, after a great deal of practice in the developed world, has been refined to either a state of invisibility through familiarity. Or one of a pantomime everyone uses to pretend to themselves that this is just the way things are, or for a minority, to ensure they remain so.

But in any case it is legal when practiced on any scale worth studying, or mapping, in the light yellow or orange countries on that map. But if it is legal then it does not appear on this map. A kind of Catch-22, or of corruption all of itself really.

When thinking of really big examples the Bush sponsored Medicare Prescription Drug Improvement Act still comes to my mind first.

An Act that on the one hand makes prescription drugs cheaper and more available to seniors enrolled in Medicare. Old lady’s arthritis pills, what is not to like there? Within a contorted system of plans and rules it should be noted.

An Act that also expressly forbids the United States Government, the largest buyer of prescription drugs in the US, from negotiating lower drug prices from drug companies. Take a moment, let that sink in. Within a very simple system it should be noted.

This seems, to me, like it might be an example of corruption.

But not according to this map.

Senator Hillary Rodham Clinton (Democrat, New York) announced that she is working on legislation to improve the Medicare drug benefit by creating a regulatory structure to find and remove less-efficient private drug plans.  She also said that the government should have the authority to negotiate for lower drug prices in Medicare (which is strongly opposed by the pharmaceutical companies) and that pharmacists should be reimbursed for filling prescriptions for Medicare beneficiaries who were unable to prove eligibility because of computer glitches; and the February 15 deadline for reimbursing states offering Medicaid coverage to address coverage gaps should be extended - Link to source

When the custom was to speak plainly I think this was referred to as price rigging. But as with IQ tests, or SAT scores, or law enforcement, test results tend to provide as comfortable outcome as possible for those designing or administering the rules or laws, and more so as time passes. Intelligence declining according to historic measures? Grades and scholastic aptitude measures in decline? Too many of your friends prevented from expressing their desires by unreasonable limits imposed by a different interest group?

Don’t cheat, don’t break the rules or laws, that is corrupt. Change them, that is democracy.

In the more specific example below, as shown in the accompanying screenshot from my unsubscribe form submission with WebMD Professional, you can see mention of this type of legal corruption that few seem to see because it is part of the fabric of life. In this case it was a piece of that fabric I had not seen before, so it was shocking.

After unsubscribing I was asked the reasons for my leaving, and other questions WebMD thought might provide useful answers.

The most interesting to me was: “Do you only participate in programs sponsored by pharmaceutical companies when an honorarium is offered”.

I did not know that it was common, and legal, practice to pay doctors to listen to drug companies make presentations, did you?

I suspect that this information is presented either alongside, or perhaps even as, research or continuing education requirement courses offering credit to doctors who have to submit evidence they are keeping abreast of academic advances in their field. After all what else but interest or compulsion would make you travel across the country and stay in a hotel with a bunch of doctors, and attend a conference and it’s presentations and discussions?

Honorarium, far more mellifluous a word than any accurate alternative. Some examples? Substitute “bribe” or “inducement” in place of “honorarium” and the question would be as accurate, and more honest.

Paying someone to attend, no mention is made as to whether honorariums on average defray, meet or exceed the expense of attending for instance. Or that they are only offered if the party paying expects to profit from the act in some way. That might be by gaining a reputation as a sponsor of worthy causes I suppose. But these are drug companies paying doctors after all, and drug companies are first for-profit, not for-health, enterprises. What do you think the odds are that this practice is so widespread that it is regulated?

How large are honorariums, how do they vary, is there any correlation between the size of the payment and the market potential of the drug in dollar terms, how large are they compared to the expense of attending in specific drug categories, and on average, etc., etc.

That would be an interesting little project.

how drug companies bribe doctors
Unsubscribe screen shot composite from WebMD “Professional”