The findings of the CDC’s own internal review committee were critical of the methods used to estimate 400,000 obesity-attributable deaths. Committee member Rachel Ballard-Barbash, who is associate director of the National Cancer Institute’s Applied Research Program, wrote: "…we should no longer be using the [CDC’s] relatively simple methodology." A second member suggested that the study was never a " ‘state-of-the-art’ attempt to estimate the health burden" of overweight, obesity, and other causes of death. A summary of these findings concluded that "the fundamental scientific problem centers around the limitations in both the data and the methodology in this area."
In terms of both data and methodology, the study by Dr. Flegal’s team of researchers is far superior to the 400,000-deaths study. The latter study relied on calculations from a 1999 JAMA study by University of Alabama professor David Allison. His study blamed obesity for 300,000 deaths in 1990. The CDC’s study increased that number to 400,000, noting: "We used the same procedure reported by Allison et al. to estimate annual overweight-attributable deaths." Dr. Flegal’s study is superior for the following reasons:
"Scientists agree that Flegal’s study is superior."The Flegal Study Used Much More Recent Data
Despite having access to more recent data, the CDC based its 400,000-deaths conclusion on information dating as far back as 1948. Allison and his colleagues relied on six population studies to calculate the mortality risk from excess weight. The average start date of these studies was 1963 and the average end date was 1983. Consequently, the 400,000-deaths estimate assumed that our ability to treat high blood pressure, diabetes, heart disease, and other illnesses linked to obesity has not improved in more than a generation.
Allison admits in his 1999 study that this is a problem. "When most of the cohort studies used were initiated," he writes, "there were fewer intervention strategies to reduce risk factors associated with obesity and fewer medical therapies for postponing death from obesity-related diseases." Despite this, Allison made no adjustment to his findings.
|Allison's Cohort Studies||Years Studied|
|Amaleda County Health Study||1965-1975|
|Tecumseh Community Health Study||1959-1985|
|Framingham Heart Study||1948-1980|
|Americans Cancer Society Prevention Study||1960-1972|
|Nurses Health Study||1976-1992|
|NHANES I Epidemiologic Follow-up Study||1971-1992|
Allison had access to much more recent data, but failed to use it. He relied on the NHANES III dataset collected by the CDC during 1988-1994 to determine the prevalence of obesity. Yet he failed to use either NHANES II or NHANES III in calculating the risk of obesity.
Allison has actually criticized other researchers for using one of the very same cohorts that he relied on for his 300,000-deaths study. In a letter published in JAMA in 2003, he wrote:
"… Peeter, etal. looked at a cohort of individuals residing in Framingham, Mass beginning in approximately 1948 … Since 1948, treatments for cardiovascular disease have improved and mortality rates have been reduced. For this and other reasons, the true effect of BMI on longevity of mortality rate may change over time."By using more current data through 2002, Flegal’s team observed a dramatic decline in the risk of obesity. In fact, the most recent data show no increased risk among those with a BMI under 35:
"Excess deaths associated with obesity (BMI >30) were calculated as 298,808 according to the NHANES I relative risks, 26,917 according to the NHANES II relative risks, or 43,640 according to the NHANES III relative risks. In all 3 cases, however, the majority of deaths associated with obesity were associated with BMI 35 and above: 186,498, 21,777, or 57,515 deaths respectively. (NHANES III relative risks produced a negative estimate for BMI 30 to <35.) For overweight (BMI 25 to <30), the data consistently suggest no excess deaths overall ... the largest difference [in deaths] is due to the inclusion of the mortality data from NHANES II and NHANES III, which decreased estimates by 63% or more relative to NHANES I mortality data alone ... Relative to NHANES I, the more recent data from NHANES II and NHANES III suggest the possibility that improvements in medical care, particularly for cardiovascular disease, the leading cause of death among the obese, and its risk factors may have led to a decreased association of obesity with total mortality. Cardiovascular risk factors have declined at all BMI levels in the US population, but, except for diabetes, the decline appears to be greater at higher BMI levels."The CDC itself has acknowledged the importance of examining newer data:
"Earlier estimates only reflected the obesity-related health risks that people experienced in the 1970s. The newer data (some with mortality follow-up through 2000) appear to reflect a real decline in the risks of dying from obesity-related diseases like heart disease. Big improvements in the control of risk factors for heart disease, such as better drug management of high blood pressure and cholesterol, may have resulted in far fewer people dying today as a result of obesity."
The Flegal Study Used Nationally Representative Data
Only one of the six population studies Allison used to calculate the likelihood of death from excess weight was nationally representative. The others overrepresented whites and Americans from upper socioeconomic classes. In 1985, a study appearing in The Annals of Internal Medicine explained:
"The databases on which height-weight tables are based have sampling and design problems … The largest databases, those of the Build studies and the American Cancer Society study, are not particularly representative of the United States as a whole. These studies disproportionately include men, whites, members of the upper and middle classes, and persons under age 60 … The Framingham Study and other studies based on community or occupational samples likewise do not represent the population."Discussing the 400,000-deaths study, the Flegal team notes:
"Previous estimates used data from a variety of studies to estimate relative risks, but the studies had some limitations. Four of the 6 included only older data (2 studies ended follow-ups in the 1970s and 2 in the 1980s), 3 had only self reported weight and height, 3 had data only from small geographic areas, and 1 study included only women. Only one dataset, the National Health and Nutrition Examination Survey (NHANES) I, was nationally representative."By using only the CDC’s nationally representative NHANES I, NHANES II, and NHANES III datasets, the researchers’ estimate of 25,814 obesity-attributable deaths better represents the American population and avoids problems associated with relying on geographically constrained samples. If Allison’s team had limited itself to only the nationally representative data from NHANES I, they would have reported 13 percent fewer deaths.
It may seem counterintuitive that overrepresenting whites and upper socioeconomic classes would increase the number of deaths, but there’s a simple explanation. Many studies show that African-Americans have little, if any, increased mortality risk from obesity. For instance, a 1999 study published in The New England Journal of Medicine found: "Black men and women with the highest body mass indexes had much lower risks of death."
The Flegal Study Controlled for the Influence of Age
It is widely recognized that older people have a lower risk of death from obesity. Allison’s method, which failed to account for age, led to an overestimation of deaths. Months before the CDC’s 400,000-deaths announcement was made, Flegal and her co-authors submitted for publication two papers that pointed to this problem. Writing in The American Journal of Epidemiology, they argued:
"Existing estimates of the number of deaths attributed to overweight and obesity were calculated by using a method likely to produce biased estimates, when the effects of obesity vary by age or other characteristics. Estimates of deaths attributable to overweight and obesity arrived at by using this approach may be biased and should be viewed cautiously."The second paper, published in The American Journal of Public Health, warned:
"Our examination suggests that given present knowledge about the epidemiology of obesity, and especially the impact of age on mortality risks associated with obesity, it may be difficult to develop accurate and precise estimates. We urge caution in the use of current estimates on the number of deaths attributable to obesity and also urge researchers to devote greater efforts to improve the data and methods used to estimate this important public health statistic."In light of Flegal’s findings, Science magazine reported in May 2005: "Allison concedes that in retrospect, his decision not to stratify by age was a mistake."
Deaths Due to Overweight vs. Deaths Due to Obesity
The study by Flegal’s team of researchers found that being overweight (BMI 25 to 29.9) may prolong the lives of more than 85,000 people each year. In fact, some of Allison’s data supports this theory.
"Allison concedes that in retrospect, his decision not to stratify by age was a mistake."While most of the 300,000 deaths claimed by Allison’s study derive from the "obese" category, some derive from the "overweight" category. But his own data show no statistically significant relationship between being "overweight" and having an increased risk of death. In fact, 17 of Allison’s 60 reported relationships indicate that being overweight has a (statistically insignificant) protective effect. The failure to report a significant relationship between overweight and increased mortality is not surprising. Most studies find little or no increased risk associated with being overweight. A separate study by David Allison found that the lowest risk of death occurred at a BMI of 27. If the overweight deaths had been excluded from Allison’s 300,000-deaths study, it would have reported 17 percent fewer deaths.
Flegal’s Study Reported the Range of Risk
Allison failed to report a level of uncertainty in his conclusion. This uncertainty is measured by confidence intervals, which reflect the range of what the real number could be. If the confidence intervals include the possibility of zero elevated risk, the risk is considered statistically insignificant. An editorial accompanying Flegal’s JAMA study points out:
"An important and possibly overlooked contribution by the study by Flegal et al is the formal calculation of confidence intervals around the estimate of obesity-related deaths. When relative risk estimates are only modestly elevated, as in the case of obesity, very small changes in the relative risk translate to large differences in the population-attributable fraction. Thus, it should come as no surprise that the 95% confidence interval around the estimate of 112,000 deaths ranges from 54,000 to 170,000, greater than a 3-fold difference reflected within the range. Although the other studies that previously estimated obesity-attributable death did not include confidence intervals, the estimates from those studies should be assumed to have underlying uncertainty at least as great."Commenting on the study in the CDC’s internal review committee report, Dr. Rachel Ballard-Barbash, associate director of the National Cancer Institute’s Applied Research Program, wrote:
"…It is common practice in medical and science journals today to require that confidence intervals be provided even within tabular summaries of data. Therefore, one might question why JAMA did not require such condidence intervals. Because casual readers tend to focus on tabular summaries, it is likely many readers had a misimpression of the range about these estimates."Keeping in mind the concept of confidence intervals and statistical significance, three points stand out:
- Flegal’s conclusion that being overweight entails a lower
risk than being a normal weight is statistically significant.
The 86,000 lives saved from overweight have confidence
intervals that range from 161,000 lives saved to 11,000
- As noted above, Flegal reported a (statistically insignificant)
protective effect of BMI 30-35 in the most
recent dataset. It is also true that the combined data
from NHANES I, II, and III show no statistically
significant mortality risk for BMI 30-35.
- The final conclusion that overweight and obesity combined cause excess deaths is not statistically significant. The number of 25,814 deaths from overweight and obesity combined has a range from 86,284 lives saved to 137,913 lives lost.