Modern Medicine and Maladies of Evidence

I won’t name any names, but I just read that a former science editor at a major, global publication has concluded that science advocacy is boring, and that she would now prefer to “slaughter the sacred cows” of conviction. Leaving aside the somewhat brutal image, and the potential innocence and genuine sacredness of the cows in question, we may simply note that acquiring conviction born of science generally takes years, even decades. Disparaging it just takes a news cycle, innuendo, and a bit of click-bait.

If we translate this ominous declaration, it readily becomes: “reminding people what is actually true (e.g., climate change) in the hope they might finally, actually do something about it is repetitive and boring. So, let’s tell them the titillating lies they want to hear instead.”

Let us pause on that note and pay our respects to the law of unintended consequences. The industrial revolution was not intended to produce epidemic rickets, or presage the inevitability of anthropogenic climate change- but it did. The advent of the Internet was not intended to make cyber-captives and/or addicts of us all, but it did. The burgeoning of social media platforms was not intended to undermine our relationship to truth (or democracy), but it has (might). All hail the law the of unintended consequences; we ignore it at our peril.

The laudable objectives of evidence-based medicine were not intended to turn the subtleties of methodology into a cudgel wielded by one faction in aggression against counterparts, repudiate the merits and essential role of sense, or undermine the very concept of expertise. But alas, in this land of the law of unintended consequences, they have done just that. The undeniable merits of randomized controlled trials (RCTs) have been corrupted from the power to inform, into a tendency to tyrannize.

I will make the case preferentially within my own purview, lifestyle medicine and nutrition. But it pertains as fully to cancer screening and lipid lowering, immunization and genomics. Wherever medical news incites passions, a contagion of unintended consequences finds vulnerable hosts. If ever we are to overcome these maladies of our own, inadvertent devising, we must first understand them.

1) One method to rule them all

A hammer is an excellent tool; every carpenter needs one. But it makes a very poor saw, a lousy screw driver, and an utterly useless router.  It’s a great tool, but it only does what it does.

The RCT is just the same, as fundamental to the advance of medical understanding as a hammer (or nail gun) is to every carpentry project. But it only does what it does. What does it do? Many of those who shout loudest about its episodic absence seem to have no idea.

For starters, RCTs only ever answer the questions that are posed; this is true across all methods of science. No method can produce useful answers to vapid questions; no quantity of science can redeem senselessness. Among the tyrannies and tirades attaching themselves to evidence these days is the notion that science can operate independently of sense; it cannot. Sense is the incubator of good questions; science then follows to pursue answers not already on obvious display.

The RCT is a robust tool with a number of strengths, almost all related to confidence about the attribution of X (an effect) to Y (a cause). Randomization is not a panacea of any kind, but offers the promise of controlling for confounders both known and unknown, and preventing allocation bias. What are confounders and allocation bias? Feel free to Google those, but let’s simply note here that everyone presuming to tell us about the primacy of RCTs should already know.

A control group accounts for non-specific, or placebo effects. Blinding forestalls the influence of many varieties of bias, ascertainment bias notable among them. Again, feel free to Google it.

RCTs have very important weaknesses, too- and especially in the lifestyle domain. Rather than belabor those here, let me just ask you a question instead: would you be willing to let someone else “randomly” assign you to a dietary/exercise/sleep pattern for the next decade or two, rather than choosing for yourself? I don’t know anyone else who would be, either. 

Basic science is generally the best source for insights about mechanisms of action. RCTs are vital to confidence in attribution, but generally for short-term effects in select populations. Our best insights about population-wide and longer-term effects come inevitably from observational epidemiology.

One method of science no more obviates all others than a good hammer precludes the utility of a screwdriver. 

2) I heard it through the grapevine

Hard though it may be to remember or believe, there was a time when the only people who read medical journals had subscriptions, and received them in print. These were, of course, carted around on sleighs, because the wheel had not yet been invented…

The audience for these incremental additions to the biomedical fund of knowledge included practitioners and scientists, and credentialed media outlets that would receive print copies in advance under embargo. Under such quaint and archaic circumstances, media coverage was, if not always spot on, generally competent and careful and considered.

Now, of course, medical studies- or at least snippets of them- are circulated digitally in real time. Everyone has access to them, and that is the only license required to opine. Opine people do, with or without relevant expertise. In fact, expertise is a hindrance, because experts feel obligated to read studies completely and carefully, and at least to attempt a bias-free interpretation. The highly partisan, non-expert knows no such constraints; and that much less so the true fanatic, fool, or Internet mercenary.

So, often uninformed opinion is the first wave. Any given faction can go shopping for the conclusion it already favored, find it, and amplify it. And then, when the as yet undecided or nominally impartial go shopping for information on the topic, they find a trove corroborating whatever opinion was espoused at the start of this chain reaction. Volume and vehemence are mistaken for veracity, and so these new recruits- some of them with professional credentials- embrace the new narrative. Their endorsement is then trumpeted by those who propounded their own wishful thinking at the start as further evidence that…they were right. And so, we have a mole hill of idle opinion and misinformation grown into a veritable mountain.

What makes a bit of misinformation into a mountain of misinformation? The claim that it stands on a bedrock of evidence. But nothing in that claim requires the evidence to have been interpreted expertly, accurately, or even honestly at the start.

The contention that saturated fat has been exonerated of crimes against coronary arteries, and that we should eat more meat, is just such a false narrative. The claim that sugar has been overlooked in dietary precautions of the last half century is just such a false narrative. Richly detailed historical accounts are overtly misrepresented. And so we have the imprimatur of evidence corrupted into the justification of something else entirely. We all now strive in the shadows cast by such mountains.

3) What’s good for the goose…gives the gander a pass

Challenges to the reliable interpretation of evidence don’t invite caution as they should; instead, they seem inevitably to encourage a rush to judgment in the opposing direction. 

Even as we are told how and why to distrust science, we are told how and why some new study is- all on its own- enough to change everything we thought we knew until this morning.

Folks, it takes less than half a wit to see that both of these cannot be true. If science tends to be less reliable than we might like, then how much less reliable is any one study, compared to the aggregation of evidence across decades? Conversely, if we can suddenly “know” the truth based on any one study dominating a news cycle, how much more so based on the full weight of evidence? If you don’t trust the staggering mass of evidence regarding climate change, or evolution, or the benefits of eating more vegetables and fruits, how can you possibly trust any one study claiming something else? The answer is, you can’t- unless you are simply shopping for support for the opinion you’ve already decided to call your own, no matter what the evidence says.

That we can’t trust ALL of the evidence, but can confidently change everything we thought we knew based on some tiny fragment of evidence is inconsistent nonsense at best. At worst, it is hypocrisy. The law of unintended consequence warns us to expect the worst.

4) Even truth… is judged by the company it keeps

Imagine that partisans of Religion A have concluded that killing in the name of god is good, and right, and holy. Partisans of Religion B have concluded, conversely, that no god worth having would sanction killing in her name.

Now imagine that an independent group of secular, agnostic, and possibly atheistic ethicists takes the matter on. They follow where the precepts of their discipline, logic, precedent, and conscience lead them- and reach the conclusion that killing in the name of god is unethical. What happens next?

The Religion B congregation swoops in to claim these ethicists as their own and celebrate their righteous conclusion. Religion A adherents, however, repudiate the ethicists as hacks, note their alignment with Religion B, and dismiss their conclusion as partisan and prejudicial.

But this dismissal obviously puts the sausage-laden cart before the horse. These ethicists had no vested interest in either religion; they simply pursued the “right” conclusion. That it aligned with the views of one group rather than another was happenstance.

So, too, for nutrition- where the consensus about the merits of plant-food-predominant diets, and the science underlying that case, is massive, global, and transcends dietary preferences, academic disciplines, or any given ideology. In other words, the conclusion is non-partisan. But it happens to align, like the conclusion of our imaginary ethicists, with the views of some very partisan groups that advocate for the same, causing groups with opposing conclusions to impugn the conclusion itself as partisan. 

*******

The goals of evidence-based medicine are laudable. But evidence is just means; understanding is the ends. These means, like all others, are subject to the law of unintended consequences. By mishandling the means, we corrupt the intended ends- and propagate a costly misunderstanding at the expense of common sense, common cause, and common knowledge. Until or unless we better manage the conscription of evidence into the contagions of misinformation, we are all apt to be victims of just such plagues.

Dr. Katz’ new book, The Truth about Food, is due out October, 2018- with all proceeds going to support The True Health Initiative. For more information, click here.