Good Nutrition, in Circles on The Road to Nowhere

Given the almost dizzying frequency of diet-related headlines, the one true revelation about nutrition is superficially the least likely: there is no real news about nutrition.

Despite constant, seemingly contentious news about nutrition, there is just about no news in the sense of something actually new that we need to know, and put to some daily use. The fundamental truths, the big picture, the stuff that has stood the test of time and applications of both science and sense- is pretty much what it has long been.

That statement is robustly true at the level of dietary pattern, but not equally true across the entire spectrum of what constitutes “nutrition.” There is still a lot we don’t know about this fatty acid or that, this vitamin-like compound, that resistant starch, or those soluble fibers. But those advances inform understanding pertinent to the supplementation of the overall diet, not substitutions that reconstitute the basic character of it. Eating, of the variety that prevails, is about foods in combinations, not nutrients in isolation, and there- despite all the news- there really is no news.

Consider, for instance, the latest high-profile paper about butter, predictably precipitating the latest spate of headlines insinuating either whatever editors think will capture eyeballs (so-called “click bait”), or the preconceived preference of the writer.

The paper itself is a meta-analysis. These are now conducted and reported so routinely in the popular press, I suspect most readers have been lulled into a false sense of security about their applicable knowledge. Do you really know what a meta-analysis is, or what it is for?

They are, technically, “for” something very particular. They are for overcoming beta, or type-2 error; a lack of statistical power. Allow me to explain.

Imagine we are concerned that some drug- say, Vioxx- despite having some desired therapeutic effect (i.e., relief of joint pain) is also doing some very serious harm (i.e., inducing the occasional heart attack). Now, imagine that the studies suggesting that harm never “prove” it because the association is not statistically significant. There are a few more myocardial infarctions in the Vioxx than the placebo group, but always within a stochastic span that might be ascribed to random variation; a fluke.

Why would sample size matter? Well, leaving aside the statistical fine points, it’s clear that if X causes Y about one time per million people per year, you are VERY unlikely to discern that in a study of ten, 100, or even 10,000 people. With one extra case per million per year, you would need a study of 10,000 people to run for 100 years to pick up just that one extra case reliably, and even then- a difference of just one case in 10,000 people wouldn’t prove a thing. You could easily imagine a 100-year study of 10,000 people in which somebody gets struck by lightning, or killed by a shark- and it would have nothing to do with the pill they were assigned.

You get the idea. When events are rare, you need very large numbers, very long studies, or both, to accumulate enough such outcomes to make sense, let alone statistical sense, of the findings.

Meta-analysis is designed to address this problem. By pooling data from multiple trials addressing the same or very similar thing, you generate the power of large numbers- and can often replace suggestions about some association with some degree of statistical reliability.

There is, therefore, a whole lot that meta-analyses are not designed to do. They cannot answer new questions, since they depend entirely on studies already run and completed. They cannot improve on the methods of the studies they pool, but rather, are held hostage by any such deficiencies. In fact, since a meta-analysis pools data from related but diverse prior studies, it tends, if anything, to be more prone to methodologic sludge than its own antecedents. Here’s what I mean.

Consider some wildly outlandish meta-analysis about…the association of butter consumption with heart disease. OK, not so outlandish after all.

To answer any questions about the specific effects of butter intake on heart disease risk requires that sine qua non of epidemiologic research: all other things being equal. To attribute heart attacks to butter, or to something eaten in the place of butter, it is first necessary to ensure they are not due to something else. If eating butter were routinely followed by a smoke, for instance, then any apparent ill effects of butter might instead be due entirely to tobacco. This is called confounding.

Preventing, or redressing such confounding is called “controlling,” and requires balancing out factors other than the exposure of interest between study groups- either when enrolling participants (i.e., randomization; stratification; etc.), or through statistical and analytical manipulations. Since meta-analyses rely on study populations long since enrolled, the only option resides in the realm of analytics.

But that option is constrained. If one study obtained dietary information using diaries or 24-hour recalls, and another using food frequency questionnaires, the data are not just flawed, but flawed in different ways. Applying the same analysis to a diversity of flaws generates a new generation of often unpredictable flaws. Do the meta-analyzed data reliably preclude some other dietary variable, or even some non-dietary variable, being the actual cause of any good or ill effects associated with butter intake? Never entirely.

And this problem is especially acute in studies of lifestyle practices, such as diet, because they tend to cluster. We have long known that health conscious people do not, for example, chase down a double-bacon-cheeseburger with a broccoli floret; they don’t eat the double-bacon-cheeseburger in the first place. People who routinely eat butter tend to differ in other ways, related to diet at least but often extending even into other lifestyle practices, from those who routinely eat olive oil, or stick margarine, instead. No observational study, let alone a meta-analysis of such studies, can fully address this.

The one thing that can- a randomized trial- brings us back to the original problem of sample size and timeline, and then tosses a few more problems on the same, weighty heap. A randomized trial about butter and heart disease would need to assign people to exactly corresponding diets differing only with regard to butter intake. That is problematic already, because something needs to take the place of butter- and so effects observed might be due to butter eaten or avoided, but might also be due to what is eaten instead. Mostly, though, the problem is simply how infrequent heart attacks or diabetes or death would be in such a trial, requiring an enormous sample size and lengthy timeline. Such a study would break the bank.

That should wrap up our digression about research methodology. The only relevant thing I can think to add is that, yes, I have conducted and published meta-analyses, and written in some detail about the relevant methods in textbooks. I respect them and think they are of genuine value, but also respect their limitations.

In this context, then, we have the new meta-analysis about butter and health outcomes.   The study basically showed a modest, positive association between butter intake as an isolated variable and all-cause mortality; and a modest, negative association with diabetes. In other words, to the extent these associations overcome the limitations noted above, they mean: a bit more butter means a bit more risk of early death, and a bit less diabetes. Translating that to “butter is back” is, obviously, one helluva stretch.

I think my colleague, and senior study author, Dr. Dariush Mozaffarian, pretty much nailed the interpretation by saying: “Overall, our results suggest that butter should neither be demonized nor considered ‘back’ as a route to good health. More research is needed to better understand the observed potential lower risk of diabetes, which has also been suggested in some other studies of dairy fat. This could be real, or due to other factors linked to eating butter -- our study does not prove cause-and-effect.” In response to this, I can pretty much only append: amen.

Lead study author, Laura Pimpin, commented that butter might be better than the white bread on which it often resides. Perhaps so, but here, I have some doubts. I suspect that if the health “effects” of just white bread could be isolated from the effects of overall dietary pattern, they would be no less ambiguous than those reported for butter.

The larger context here extends beyond butter to the on-going debate about saturated fat in general, and associated health outcomes. The matter has, in my view, been badly misrepresented. Arguments “for” saturated fat routinely quote two meta-analyses, one from 2010, the other from 2014, which showed comparably high rates of heart disease at the low and high ends of the population scale for saturated fat intake. For whatever it’s worth, that scale was quite narrow; saturated fat intake did not really vary all that much.

But across the range of variation that was observed, heart disease rates did not vary. This in no way indicates that saturated fat or its usual dietary sources are “good” for us now, since high rates of heart disease did not decline as saturated fat intake went up. They stayed the same. All this really means is that whatever people are eating instead of saturated fat appears to be almost exactly as bad; there appears to be more than one way to eat badly.

Just that has since been shown. Neither the 2010 nor the 2014 meta-analysis looked at what people were eating instead of saturated fat, or vice versa. But a study in 2015 did so in roughly 85,000 people. The findings were wonderfully concordant with the prior weight of evidence, and what we thought we knew on the basis of it. When saturated fat was replaced by trans fat, things got worse. When it was replaced by sugar and refined starch, as it often has been, things were comparably bad both times (i.e., the adverse health effects of added sugar and of saturated fat from meats and dairy look to be almost stunningly commensurate). However, when saturated fat calories were replaced with whole grain calories, and/or unsaturated fat calories from nuts, seeds, avocado, olive oil, fish or seafood, rates of cardiovascular disease declined significantly.

And then, hot off the presses is an epidemiologic study in roughly 125,000 people followed for approximately 30 years. The study found increases in all-cause mortality with increases in both saturated fat and trans fat intake; and decreases in mortality with increases in both polyunsaturated fat intake, and monounsaturated fat intake.

The recent cohort studies have their own limitations, of course, and are not the ideal kind of science to prove cause and effect. But they also offer a number of advantages, the most notable of which is attention to the critical and often-neglected question in nutritional epidemiology: instead of what? Butter is clearly a better choice than stick margarine made with trans fat, something we have known for literal decades. It is not nearly as good a choice as olive oil.

Which means the entire debate took us around a circle to deposit us more or less back where we thought we were at the beginning: eating mostly vegetables, fruits, whole grains, beans, lentils, nuts, seeds, and water when thirsty is apt to promote our health. Olive oil is good for us. Eating more butter is not. Once more around we go, and we wind up somewhere remarkably concordant with not just the weight of scientific evidence, but also sense, and the simplicity of a time-honored understanding. That this same dietary emphasis is vital to the fate of the planet cannot be repeated too often.

This column is a lamentation, I suppose. After all, while there is much to learn about the details of diet and health, there is little argument that using what we have long known could help eliminate up to 80% of the current, global burden of chronic disease and premature death associated with it. I am lamenting this needless, on-going loss of years from lives, and life from years.

The frequent hyperbole and non-news about nutrition seems to have us forever going in circles, rather than making actual progress and putting what we know to any good use. Worse, the competing contentions generating constant headlines threaten to run the common understanding of eating well off the rails altogether. All the more so given how often those rails are liberally greased with whatever culinary emollient- butter, margarine, this oil or that- has us fascinated for the current news cycle.

Until or unless we accept that on-going nutrition research is designed to add to and refine the fundamentals of what we already know, not replace them every 20 minutes- I recommend a scopolamine patch and a well-fastened seatbelt for this never ending, dizzying ride taking us…nowhere.