I won a debate this past week. I was pleased to win- despite a crowd mostly hostile to my position at the start, and frequent shifts by my opponent from both reason and the stipulated resolution to derisive innuendo and outright aspersions hurled at my character- because the ostensible topic, food and health, is of enormous importance. That topic is more than ample reason in its own right to weather the slings and arrows of iniquitous confrontation, and take one for the team. How we eat has implications for countless years in countless lives, countless life in countless years, and the fate of the planet.
But the debate was actually far less about diet and far more about evidence, and the proprieties of science: how it is conducted, how it is interpreted and applied. And that has greater ramifications still, pertaining to every important matter contacting and shaping the human condition, from climate change to environmental toxins, from communication to elections and the fate of democracies, from entertainment to suspension bridges, from vaccination to antimicrobial resistance, from space travel to streaming music, from stem cells to synthesized meat, from in vitro fertilization to how we fertilize the fields of sustainable agriculture. Our entire modern culture, and our collective fate, are suffused by science.
The misunderstandings of science that prevail and roil us are protean, and potentially calamitous- in general, and in concentrated fashion in my own particular space, the applications of diet and lifestyle to the promotion of health and the prevention of disease. Let’s dive in, the light of proprieties in hand, to see if we might illuminate the murky depths.
To begin, no one study changes all that we know. Science is profoundly, ineluctably incremental. We may all celebrate the occasional epiphany of insight, but even these are all but universally buoyed to the surface by underlying currents of understanding. Even to infer, we must know. The bedrock of great conjecture is solid knowledge.
But anything approximating epiphany is rare. The progress of science is far more often genuinely mundane. The gears turn, relentlessly and more often than not reliably, to give us each day our daily grind of new understanding. A travesty of modern science abuse is the translation of this process into sequential headlines mimicking Ping-Pong, each report reversing and refuting the last. This serves the aims of media executives seeking to comfort the afflicted, afflict the comfortable, and use perpetual confusion and the promise of understanding after tomorrow’s segment- to maximize viewership. It does not serve understanding, or public health.
No one study design can answer all questions, and every design has both limitations and strengths. This is well known to all who toil in the service of science, designing the best studies we can, conducting them in the strictest possible fidelity to protocol. But it, too, is fodder for self-aggrandizing abuse in the hands of the uninformed, the foolish, the fanatic, the inveterately contrarian, or the disingenuous.
In particular, the case is made by the cabal we may summarily call “nutri-contrarians” that we only ever know by virtue of randomized controlled trials. Before saying what we can and do learn from RCTs, let me point out what we, obviously, cannot. We are interested – or should be- in the effects of dietary patterns and lifestyle practices across entire lifespans, and more- generations. We are interested in lifelong vitality, and longevity- the fusion I have labeled “vigevity”- years in life, and life in years.
But none of these crucial outcomes can be derived from a RCT. We cannot randomly assign people to a diet for a lifetime, let alone a span of generations. Only observational data will ever inform our understanding at such scope, and scale, and tenure.
But the other flaw of RCTs is less obvious, and so- more pernicious. Yes, they are a strictly controlled, rule-replete methodology. In contrast, observational epidemiology is far messier. But the entropy of messes, while challenging reliable interpretation, is also a robust defense against manipulation; researchers with questionable agendas cannot readily “game” the system when the system is what happens over years and decades to tens or hundreds of thousands of people.
The strict controls immanent to RCTs are in the hands of those running them, who thus have abundant opportunity for manipulation and mischief. There are many minor transgressions populating this opportunity, but here’s the most common and damaging: a willfully bad question. Yes, RCTs are powerful means of answering questions- but they only ever answer the questions we pose. We may ask them to compare a very bad low-carb diet to a very good low-fat diet, or vice versa; we may ask them to tell us the health effects of a vegan diet, but it is we who tell them that the vegan diet in question is comprised entirely of Coca Cola and cotton candy. RCTs can be “gamed” in a whole variety of ways large observational studies cannot.
This is not to say that observational research is itself devoid of flaws; they are just different flaws. Less prone to manipulation (although by no means immune, absent relevant safeguards), observational research is also less subject to controls. There is always a bit of mayhem in the mix; it’s always possible that while both X and Y happened a lot, they are true, true, and unrelated.
What’s the remedy? A mix of methods, the very argument colleagues and I make in a paper currently under peer review for publication. In that paper, we focus on the needs of lifestyle medicine, my purview, for data spanning all aspects of vitality, and the full life span. We call the method we espouse HEALM: hierarchies of evidence applied to lifestyle medicine. We conducted a methodologic systematic review to reach our conclusions, and those conclusions are generalizable.
In brief, we contend that various assemblies of diverse research evidence can “add up” to the same level of confidence or conviction. Applying an approach we call evidence threshold pathway mapping (ETPM) to get to that standard, we detailed the specific contributions of various methods. Basic science studies, often conducted in cell culture or animal models, cannot tell us what happens in people; but they can tell us whether something can work, and how it does. They contribute mechanistic insight, a key piece of total understanding. The things we know best, we know both their what, and their how.
RCTs, and related intervention trials, make their signature contribution as well; they contribute attribution. The strict controls of RCTs allow us to say with considerable confidence that X and Y did more than happen in common context; they allow us to conclude that Y actually caused X.
But this causal contribution more often pertains to a short-term phenomenon, such as change in lipid levels, or glycosylated hemoglobin levels, or blood pressure. Far less often do people stay randomized long enough for us to compare decisive outcomes, such as death rate.
How, then, to know best? Hybrid vigor. Mechanistic studies to tell us whether, and how. Observational studies to point out patterns at scale that matter most: who lives, who dies, when, and how. RCTs and related intervention studies to tell us whether what happened in those observational patterns is concordant with cause-and-effect pathways in the shorter term. When we can aggregate these- a clearly elucidated mechanistic pathway; a strong and important association at scale; and confirmation of attribution- we have a robust basis for knowing. When all such sources are abundant and replicated multiple times in independent labs, and when that entire pattern is directionally consistent- we are in, or near, the neighborhood of established fact.
And no, some contrarian view or discordant RCT, will not suffice to alter that. Maybe it will be a reason to look again, and carefully; or maybe it is simply misguided or willful nonsense that should be dismissed. But true understanding and facts are hard-won; they do not surrender to a news cycle of hyperbolic headlines, and you should not surrender them so readily either. Science is cumulative and weighty. When dismissals of its accumulated mass are not simply self-serving, they are generally wifty, and wayward.
Be wary of common traps. Those seeking to fortify the opinion they already own, rather than truth, will selectively point out flaws in studies based only on their outcomes. This is sadly common, among my friends and foes alike. In the debate, my counterpart ridiculed observational epidemiology and trend data, until those same sources helped make a damning point- and suddenly, they were gospel. This is rather flagrant chicanery, and worse- hypocrisy. What’s good for the statistically fastidious goose must also be good for the methodologically meticulous gander.
Look out for reverse causality, hiding in plain sight. I may tell you good diets lower LDL cholesterol, and lowering LDL reduces cardiovascular events and premature deaths. But an opponent could say: I know of studies where lower LDL is associated with higher death rates! Yes, that’s true- because serious illness causes LDL to fall, and it causes death. So, LDL is not the reason for mortality in such context; it is a consequence of life-threatening illness. All scientists worth the salt in their circulations know of such perils, and account for them. But the merchants of doubt do not, or pretend they do not.
Speaking of merchants of doubt, let’s not let them talk us out of what we know from simple observation and pattern consistency. That’s just what they attempted to do with tobacco. We have no randomized trials to prove the harms of tobacco; we know them reliably just the same. We know many other things besides: that water puts out fire when gasoline does not; that apples tossed in the air fall back down rather than floating away. The contention that RCTs are the source of all understanding is propagated by those who understand neither RCTs, nor…understanding.
One final point, and another I’ve made before: if you choose to doubt the prowess of science while accessing my commentary, or anything else, in cyberspace you are-forgive me- a hypocrite. You cannot very well use the orchestration of obedient electrons the scientific method has gifted you as a means of decrying that method without practicing hypocrisy. A “tweet” used to disparage the reliability of science is an absurdity.
That is not to say science is infallible. It fails often, on the way to its successes. It learns relentlessly, and auto-corrects impassively, and implacably. The arc of science is long, but it bends toward truth.
The proprieties of debate are, perhaps, debatable- mutable by place, persons, and platform. The proprieties of science are not.
All tools can be used well or badly. Science is the most awesome power tool humans have conceived to extend our reach beyond our grasp. Used well, it can help us construct a better future. Used badly, it can mangle understanding and dismantle its very foundations.
Ingenuity sufficed to put the power of science in our hands. It requires wisdom, and that most uncommon of things- sense- to use it well.
Dr. David L. Katz is the founding director of Yale University’s Yale-Griffin Prevention Research Center, where he has conducted and overseen diverse clinical research, including dozens of RCTs, for 20 years. He is a 2019 James Beard Foundation Award nominee in health journalism, and author most recently of The Truth about Food. All book proceeds go to support the True Health Initiative, a federally authorized 501c3 non-profit.
Leave your comments
Post comment as a guest