On May 25, 2005, researchers from the Harvard School of Public Health (HSPH) hosted a symposium to critique the Flegal study. Trying to maintain a state of public panic, they called her results "misleading" and her methodology "flawed," offering a number of theoretical arguments suggesting that Flegal had underestimated the true risk of obesity. They did not, however, acknowledge that Flegal anticipated and responded to their very critiques in her original paper, writing in JAMA that the HSPH arguments "did not have a major impact on our estimates of excess deaths."
The HSPH researchers argued that Flegal’s study failed to account for the influence of smoking on obesity-related mortality. They suggested that smokers are on average thinner than non-smokers, and at the same time at a much higher risk of premature death. Thus, according to their argument, by including smokers in her study, Flegal didn’t incorporate the true health advantages of being thinner. The HSPH researchers suggested a similar line of reasoning for those with chronic diseases. These people tend to be thin, but also unhealthy. In their words:
"But determining the precise range of BMI associated with lowest mortality can be difficult, because the approach that researchers use to conduct their analyses can bias their findings. One such problem is a phenomenon that researchers call ‘reverse causation’: Low body weight often results from chronic disease, rather than being a cause of chronic disease. The weight loss may have been unintentional as a result of the underlying disease process; or the weight loss may have been intentional, because patients with serious conditions often become motivated for the first time to lose weight. Regardless, because of this phenomenon, people with a BMI below 25 are a mix of healthy individuals and those who are ill and have lost weight due to their disease. Leaner people are also more likely to smoke than their heavier counterparts. If researchers fail to account for both reverse causation and the adverse effects of smoking, they will find artificially inflated mortality rates among lean people, thus diminishing the harmful impact of overweight and obesity."To address these issues, the Harvard researchers offered three solutions. First, they suggested limiting the sample to "never smokers." Second, they wanted to exclude anyone with a preexisting chronic disease. And third, they advocated excluding "early deaths" to further account for the effect of chronic disease. Dr. Flegal’s final figures did not incorporate these elements. In her original JAMA paper, Flegal explained why:
"To examine whether the increased relative risks at lower BMI levels might be related to possible weight loss associated with illness and increased mortality, which could also have decreased the relative risks associated with overweight and obesity, we repeated analyses excluding the first 3 or the first 5 years of deaths and found little change in the relative risk estimates (data not shown). We also repeated analyses including only individuals who never smoked and found that the elevated relative risks for the lowest BMI category persisted and that other relative risks were not systematically different."Flegal expanded on this point in an article published on the CDC’s National Center for Health Statistics Web site:
"If these biases due to illness-induced weight loss or residual confounding by smoking or prevalent illness at baseline had been operating, one would expect to see a lower relative risk for underweight (closer to 1) and a higher relative risk for overweight and for obesity after controlling for baseline health status, smoking, and early deaths. However, the relative risks did not follow this pattern. Thus, these analyses of the effects of exclusions and of stratifi-cation by health status for the combined data set did not suggest that the results for the full data set were affected in any important or systematic way by residual confounding due to smoking or to prevalent illness at baseline. These analyses did not suggest that Flegal et al (2005) had overestimated the risks associated with underweight or underestimated the risks associated with overweight or obesity."In other words, Flegal crunched her data just as the HSPH folks would have liked, and found that it made no real difference. She still found that being technically "overweight" had a protective effect (relative risk below 1.0):
"For ages under 70, the relative risks for overweight varied from 0.40 to 0.91. For never-smokers ages 25–69 years in excellent or very good health at baseline, after excluding the first 3 years of deaths, the relative risk for overweight was 0.45 (significantly below 1.0). For grade 1 obesity for younger never-smokers, regardless of baseline health status and regardless of excluding the first 3 years of deaths, the relative risks were always below 1.0."In a follow-up letter published in JAMA responding to Harvard’s critique, Flegal wrote:
"The relative risks were little affected by such exclusions, and changes were often in the direction opposite to those anticipated by Willett et al."Is Flegal’s result the product of some "flaw,"as the HSPH academics assert? Research from Loyola University backs Flegal up on smoking:
"Reanalysis of the Framingham Heart Study data does not support the hypothesis that there is an interaction between smoking and measures of obesity. Moreover, the estimated BMI or MRW at the minimum risk of death was similar for men and women smokers and nonsmokers alike."Researchers from Johns Hopkins and the Obesity Research Center arrived at a similar conclusion:
"Both models [smokers and never smokers] are virtually identical because of the negligible effect of smoking in terms of both magnitude and significance ... This [study] indicates that the relationship between BMI and mortality was not significantly different between never smokers and ever smokers."As for the claim that Flegal should have excluded early deaths, a frequently cited report surveying data from 29 other studies and more than 1.9 million subjects concluded:
"...either pre-existing disease does not confound the BMI-mortality association or eliminating early deaths [as Harvard suggests] is inefficient for reducing that confounding ... these results suggest that [excluding early deaths] may not be advisable because, to the extent that it has any effect at all, the magnitude of this effect is minimal, of questionable clinical significance, and of ambiguous meaning."The debate between the HSPH and Flegal is not new. An HSPH letter responding to a Flegal study in the American Journal of Public Health (published months before her better known study from April 2005) read in part:
"One obvious logical problem is that, although most people die after 75 years of age, it is cumulative obesity exposure rather than weight at a specific older age that contributes the most to the higher mortality rates associated with obesity ... Thus, the relative risks calculated from the oldest age groups do not reflect the true long-term impact of obesity on mortality."To which Flegal responded in her own letter:
"[The Harvard researchers] speculate that the number of deaths attributable to obesity in the United States may be underestimated when relative risks are calculated on the basis of current body mass index (BMI). They cite no data to support their speculations, but instead invoke the notion of ‘reverse causality.’ They hypothesize that relative risks are lowered by obese people who become ill, lose weight because of this illness to become normal weight, and die shortly thereafter of the underlying illness, surviving just long enough to be included in the study. However, this reverse-causation hypothesis is unlikely to be the correct explanation for the lower relative risks in the elderly for 2 reasons: first because data show that exclusion of preexisting illness has little effect on relative risk estimates, and second because weight loss from obesity to normal weight is relatively uncommon."An entirely separate argument made by the HSPH researchers is that data used in the Flegal study did not provide for a long enough follow-up time on individual subjects. They insist that a shorter follow-up period doesn’t capture the full consequences of obesity. But once again Flegal anticipated and responded to this point in her original study. While suggesting that the issue requires further research, she wrote:
"To examine whether the higher relative risks in NHANES I might be due to the longer follow-up in NHANES I, we compared the relative risks from the first phase of NHANES I through the 1982-1984 follow-up with the relative risks from NHANES II and III. Thus, the followup period was similar for all surveys (10 years for NHANES I, 14 years for NHANES II, 9 years for NHANES III). The NHANES I relative risks over the first 10 years of follow-up were higher in almost every BMI-age subgroup than were the relative risks from the other surveys (data not shown). Thus, even after controlling for length of follow-up, NHANES I tended to have higher relative risks than the other surveys ... [T]he relative risk for total mortality in weight-stable individuals in the latter part of the NHANES I follow-up were similar to relative risks in the earlier follow-up period."Flegal also pointed out that even in the Allison study, which predicted a much higher risk of death from obesity, "Across the 6 cohorts used by Allison et al, there was no relation between the length of follow-up in a cohort and the relative risk in the cohort."
In sum, the HSPH critique is speculative at best, contradicted by many previous studies, and does not apply to Flegal’s data. In her JAMA study she wrote, quite appropriately:
"We undertook additional analyses to examine whether our estimates of excess deaths might have been affected by factors such as length of follow-up, weight stability, weight loss caused by illness, or smoking status ... Taken together, these analyses suggest that differences in length of follow-up, weight loss because of underlying illness, or confounding by smoking status did not have a major impact on our estimates of excess deaths."