WHY CAN'T WE REPRODUCE BIOMEDICAL RESEARCH?
"Credibility is everything for science, and it is built over time in both obvious and subtle ways. Within the last 12 months, the reproducibility of science, a lynchpin of credibility, has come under intense scrutiny, both from the NIH and other government funding bodies, as well as in the lay and scientific press." Dr. Emilie Marcus (Yale University / Salk Institute) CEO of Cell Press and Editor-in-Chief of Cell, from the November 2014 issue of Cell (Scientific Credibility and Reproducibility)
"Unfortunately, in the past few years, many studies have reported that the majority of results within biomedical research cannot be replicated." From one of Harvard University's blog posts (Reproduce or Bust: Bringing Reproducibility Back to Center Stage) by Steph Guerra
"It can be proven that most claimed research findings are false." Dr John Ionnidis from the study mentioned directly below
When Dr. Marcus in the first quote above says that "credibility is everything for science," she is only partially correct. Credibility is everything for everything. Whether you are running an organic ranch (HERE), building houses, educating our nation's children, running a medical clinic, or (ahem) serving in government, credibility is paramount. Without it, nothing works properly and nothing or no one can be trusted. A society without credibility is a society in decay. So; what about the credibility of our scientific and biomedical research communities?
What is the one thing that makes "science" scientific? Although you will find about a million slightly different definitions, the one thing --- the maxim if you will --- the property that makes science scientific is reproduciblity. Reproduciblity is essentially the same thing as (or at the very least, intimately related to) falsifiability --- the various methods of testing a hypothesis to learn whether or not it is true or false; accurate or inaccurate. The whole point of the science laboratory is to remove variables so that experiments can be duplicated over and over again to make sure that they are accurate and that further science is built on sound principles. When scientific experiments cannot be reproduced either by other teams of scientists or by the scientists that did the experiment in the first place, it's not science.
The famous medical doctor, JOHN IONNIDIS, said it best via the title of the study he published in PLoS One a dozen years ago next month --- Why Most Published Research Findings Are False. I've shown you why this is on many levels, one of the most recent being just a few weeks ago (HERE). Today we are going to tackle yet another aspect of EVIDENCE-BASED MEDICINE that proves exactly what Ionnidis stated well over a decade ago; that for any number of reasons, biomedical science cannot be trusted to be true or accurate.
This problem irreprocudibility is so pervasive in the scientific community that Wikipedia actually has an entry for it called "Replication Crisis" that says, "Scientists have found that the results of many scientific studies are difficult or impossible to replicate on subsequent investigation, either by independent researchers or by the original researchers themselves. The crisis has long-standing roots. Since the reproducibility of experiments is an essential part of the scientific method, the inability to replicate the studies of others has potentially grave consequences for many fields of science in which significant theories are grounded on unreproduceable experimental work." In other words, what you are going to learn today is that the VERY FOUNDATIONS OF MODERN PHARMACEUTICAL-BASED MEDICINE are based largely on misinformation. Whether this misinformation is deliberate (outright lies and fraud) or accidental, is something you'll have to decide for yourself.
For the record, this problem is not confined to medical research. The same thing is happening in other fields as well. For instance, the website of the Federal Reserve contains a study called Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say 'Usually Not'. After looking at an awful lot of research, the authors concluded that, "Because we are able to replicate less than half of the papers in our sample even with help from the authors, we assert that economics research is usually not replicable." We see the same thing in the field of chemistry. Listen to what Dalmeet Singh Chawla said in the March 2017 issue of Chemistry World (Taking on Chemistry's Reproducibility Problem).
"A survey of over 1,500 scientists conducted by Nature last year revealed that 70% of researchers think that science faces a reproducibility crisis. Over half, however, still have faith in published literature in their field – with chemists being amongst the most confident despite reporting the most difficulty replicating other researchers’ or their own work."
Why is this such a big deal? For the very reason I mentioned to you earlier --- that modern medicine is based on the field of chemistry. And while I can see where some experiments might be tougher to duplicate because they involve living organisms and there are inherent differences in living organisms one to another, chemistry is just that; working with non-living chemicals. Nothing should be easier than reproducing experiments with non-living chemicals. Not only is this not the case, but the icing on the cake is the fact that even though scientists are often getting it wrong, the quote above shows that they think they are getting it right. Allow me to show you some of the sources revealing just how wrong we've been getting it for the past half century.
IT'S MONEY THAT MATTERS
Although the Nature survey revealed "why" our biggest and most important scientific experiments and trials can't be replicated / duplicated, the bottom line is money. If you have thumbed titles of my dozens of posts on EVIDENCE-BASED MEDICINE, you are already aware of this. How much money are we talking about? According to the website of our government's National Institutes of Health (NIH), research is big business. "The NIH invests nearly $32.3 billion annually (2016 stats) in medical research for the American people. More than 80% of the NIH's funding is awarded through almost 50,000 competitive grants to more than 300,000 researchers at more than 2,500 universities, medical schools, and other research institutions in every state and around the world." Listen to what Dr Danielle Teller (Pulmonologist and intensive care specialist as well as a past faculty member at both Pitt and Harvard) says in the Jan 2016 article, Nearly All of Our Medical Research is Wrong.
"Academic medical research functions as a gargantuan cottage industry, where the government gives money to individual investigators and programs--$30 billion annually in the US alone—and then nobody checks in on the manufacturing process until the final product is delivered. The final product isn’t a widget that can be inspected, but rather a claim by investigators that they ran experiments or combed through data and made whatever observations are described in their paper. The quality inspectors, whose job it is to decide whether the claims are interesting and believable, are peers of the investigators, which means that they can be friends, strangers, competitors, or enemies."
Dr. Sarah Weil from her Feb 2014 article Why Biomedical Research Has A Reproducibility Problem shows how this whole thing is related to the "Invisible & Abandoned" research problem I have talked about over and over on my site. "Unfortunately, scientists are typically evaluated based on the number of papers they have published and the quality of the journals in which they have published, but not on whether their findings can be reproduced. The “publish or perish” culture drives researchers to dig for significant results they can publish, and in the process may create subtle biases to report results in a manner that inflates the importance of a study and, by proxy, its authors. Whole sets of experiments that do not fit squarely with a hypothesis may be omitted from the published work to make the findings seem more convincing." Why is this such a huge issue for the American taxpayer? Maybe because of the 32 billion being spent by our government, the huge majority is paying for research that cannot be replicated. The June 2015 issue of PLoS One (The Economics of Reproducibility in Preclinical Research) showed just how bad things really are.
"Low reproducibility rates within life science research undermine cumulative knowledge production and contribute to both delays and costs of therapeutic drug development. An analysis of past studies indicates that the cumulative (total) prevalence of irreproducible preclinical research exceeds 50%, resulting in approximately US$28,000,000,000 (US$28B)/year spent on preclinical research that is not reproducible—in the United States alone."
Holy cash-cow batman! That's a lot of dough! As you might of guessed from what you've seen thus far, nowhere is this research a bigger money-maker, not to mention a bigger producer of unduplicatable studies, than is the field of CANCER RESEARCH. Think about this when you hear 'BROTHER JOE' or anyone else for that matter calling for yet another cancer-curing "MOONSHOT". The problem is, despite all the rhetoric about stopping this problem --- or for that matter, even slowing it down, it continues picking up steam and crushing everything in its path; sort of like a cartoon snowball rolling downhill.
IS ANYTHING BEING DONE ABOUT IT AND IS
Dr. Schierling completed four years of Kansas State University's five-year Nutrition / Exercise Physiology Program before deciding on a career in Chiropractic. He graduated from Logan Chiropractic College in 1991, and has run a busy clinic in Mountain View, Missouri ever since. He and his wife Amy have four children (three daughters and a son).