What is True? The Evidence Base for Evidence Based

To know that something is true - not to have faith in it, wish for it, hope, or believe it to be true, but to know - perhaps we can agree that you need some kind of evidence. To know something is true with a very high level of confidence, perhaps we can agree further that you need high-caliber, genuinely convincing evidence. Hold that thought while we toss an apple.

If you toss an apple down here on earth, it will fall back down rather than float away. I know that, and am confident you do as well. Perhaps you’d agree that we both know it with something approximating complete certainty. But how?

We’ve just established that knowing something is certainly true with anything near to certainty requires evidence. So, have you read any randomized trials about apple tossing lately? Seen the new meta-analysis?

Neither have I. We both know everything we need to know about the fate of tossed apples from nothing more than experience- our own, and everyone else’s - and the consistency of a pattern. We have never seen a tossed apple do anything other than fall back down, and absent smoke, mirrors, and legerdemain- we don’t know anyone else who has, either. Tossed apples always fall, so we wind up supremely confident the next will, too.

How confident, exactly? Well, it’s a bit hard to say. But surely millions of apples have been tossed or dropped from trees in our lifetimes alone- and so far as we know, none has ever floated off. If we substitute just one million for these countless millions, and build in even more doubt than we feel by allowing for one apple out of each million to float rather than fall, we are still 99.9999% sure the next tossed apple will fall.

In contrast, the conventional threshold in biomedical research for declaring something “significant” is 95% confidence (i.e, the famous “p<0.05”). This means we are as much as 50,000 times more confident about apples falling on the basis of mere observation than we might be about any given biomedical “truth” predicated on randomized controlled trials and meta-analyses.

This simple, and even silly, example serves, I trust, to make a point: there is more than one way to know what is true. 

Most of what we all know- including some quite important items like what not to use when putting out a camp fire- is a product of patterns and observation. This is no way obviates the unique value of randomized controlled trials (RCTs), but it reminds us that they are just one of many tools in the service of understanding what is true.

The Case for Venn Diagrams

Here, as ever, the best tool varies with the job. Consider that in 2014 I published a commissioned review paper entitled: “Can We Say What Diet is Best for Health?” Among my conclusions was that no answer was directly available, nor ever would be, from RCTs or meta-analyses. Why never? Well, “best for health” should mean for a lifetime, so ask yourself if you would let researchers randomly assign you to some particular diet for the rest of your life. Your answer, and many other reasons, are why lifelong randomized trials are all but non-existent.

Even so, I felt I could go a long way toward a decisive answer about best diets by drawing from multiple sources of evidence. The Venn diagram below conveys the gist, where “A” could be evidence of mechanistic pathways from basic science; “B” might represent relatively short-term health effects, such as changes in blood pressure, blood lipids, blood sugar, inflammatory markers, or even heart attacks in human intervention studies including RCTs; “C” might represent observational studies over long periods of time (i.e., lifetimes and even generations) in large populations; and “D” might represent studies of environmental impacts, which in turn have massive implications for human health (i.e., there are no healthy people on an uninhabitable planet…). The overlap, in yellow, is informed by all of these sources, and represents the fundamental truths of eating for health we can know with very considerable confidence.

Nutrition, News, and Newton

This matters more than may immediately meet the eye because of the confluence of two laws: Newton’s third law of motion, and the law of unintended consequences. Whatever little you may know of Sir Isaac, you doubtless know his third law: for every action, an equal and opposite reaction. Newton was no psychologist, and was talking about blocks of flotsam and their collisions, not human behavior. But that’s where the law of unintended consequences kicks in. 

Running on a mantra of “comfort the afflicted, afflict the comfortable,” the news media thrive on provocation and titillation- not understanding. That often involves highlighting equally hyperbolic, opposing conclusions from one news cycle to the next. Human behavior seems perennially inclined to grab hold of such flotsam and hitch a ride, in nutrition and many other domains. Where Newton, nutrition, news cycles, and the law of unintended consequences converge, we find ourselves forever debating contrived doubt, traveling in circles, going nowhere.

Overcoming and Understanding

Among the ways out of this impasse is clarity about what constitutes reliable evidence, how and when we can reach conclusions with confidence. RCTs will only ever be a bit of the answer when questions pertain to lifelong health effects. Other sources of understanding must get their due if we are to overcome pseudo-confusion.

Colleagues and I just published the culmination of many months of work accordingly, introducing a method- evidence threshold pathway mapping- that stipulates the following: there are various assemblies of different kinds of evidence that can produce the same level of confidence about what is true. The right assembly to produce a reliable answer will vary with the question. We then went on to introduce one such assembly specifically adapted to such questions as “what diet is best for health?” That tool we call HEALM: hierarchies of evidence applied to lifestyle medicine.

Lifestyle medicine is and isn’t different; the requirements of understanding what’s true are the same in every domain. But the sources of understanding vary by context, just as the right tool varies with the job. Recall that life-long RCTs are extremely elusive; remember the fate of flying apples; and consider how rarely you reach for gasoline when putting out a fire. Beware those telling you exactly the one way to know what you do and do not know; they are generally peddling something other than truth and understanding.

Chew on evidence carefully before swallowing, as you would an apple. But always eschew dogma- the evidence is clear that it is not part of any healthy diet.