Studies linking diet to health need to get much better

The researchers created a star-based metric that rates the quality of evidence for an association between a behavior — such as eating red meat — and a particular health outcome.Credit: Education Images/Universal Images Group/Getty

Does eating red meat shorten life expectancy? Some researchers certainly think so. Work such as research on the global burden of disease, injury and risk factors1 has prompted the World Health Organization and the US Department of Agriculture to advise people to limit their consumption of unprocessed red meat to protect themselves from diseases such as type 2 diabetes and various cancers.

Other researchers are less sure. Targets for red meat consumption set by public health officials and expert groups vary widely, with some advising people to eat no more than 14 grams per day and others not specifying a recommended limit. This sends a confusing message, which in itself is not good for public health.

It’s not just red meat: the evidence base around many nutritional and wider health advice is similarly contested. Now, a new approach may help health policymakers better assess the quality of studies assessing potential health risks. A team from the Institute for Health Metrics and Evaluation (IHME) at the University of Washington in Seattle created a star-based metric that assesses the quality of evidence for an association between a behavior — such as eating red meat or smoking — and a specific health outcome.2. A score of five stars means that the relationship is clearly established; a star means that either there is no relationship between the two factors or that the evidence is too weak to draw a firm conclusion.

What the researchers call a “burden of proof” analysis does not by itself clear up vexed questions like the risks of red meat or the benefits of vegetables. But as a judgment of the quality of available research, it can help flag up for study funders areas where better evidence is needed for firmer conclusions.

How is the star rating constructed? What are its parameters—and can the methodology itself be considered rigorous research? The IHME team did several things to try to quantify the effects of various biases in the evaluated studies. An epidemiological study, for example, may be biased in different ways than a study testing the outcomes of health interventions. The researchers also removed what can be a common source of research bias, namely the assumption that health risks increase exponentially with the parameter being studied, such as blood pressure or consumption of unprocessed red meat. And they tried to account for the bias that can occur when sample sizes are small.

Applying this framework to studies assessing a total of 180 questions produced results that were mostly unsurprising. Studies evaluating the relationship between smoking and various cancers, for example, earn a five-star rating3. Similarly, high systolic blood pressure—the force exerted by the heart to pump blood—has a five-star relationship with the narrowing of blood vessels called ischemic heart disease4.

Studies evaluating diet and its health outcomes received significantly lower scores. The IHME analysis, for example, found only weak evidence of an association between eating unprocessed red meat and outcomes such as colorectal cancer, type 2 diabetes and coronary heart disease5. He found no link in studies that examined whether eating unprocessed red meat led to two types of stroke. There is stronger but not conclusive evidence that eating vegetables reduces the risk of strokes and ischemic heart disease6.

In some cases, lower star ratings may be due to the size of the effect: for example, any health risks from eating red meat are likely to be small compared to the enormous impact that smoking has on the body. Above all, the lower-rated findings indicate that studies in these areas need to improve if they are to produce conclusive results.

It is difficult to distinguish the effects of a single dietary component from those of a complex variety of exposures throughout a person’s lifetime. Larger studies, with a diverse set of participants and strict control of their daily diet, will be needed. Such studies will lead to collaboration between research groups with different backgrounds and access to participants in different ecological settings, a move that funders should encourage. This is an endeavor worth prioritizing. A small risk to an individual does not mean a small impact on public health: a low-risk behavior can have a large impact on a population if it is very frequent.

The literature on responsible research and innovation emphasizes how metrics in science must always be checked for reliability and rigor. Broad consultation is needed and, as far as possible, the unintended consequences of the use of indicators should be anticipated, as shown by initiatives such as the San Francisco Declaration on Research Evaluation and the Leiden Manifesto. This work needs to come sooner rather than later.

We have evidence that underpowered clinical trials, lacking the necessary controls to make sense of the data, do not help. If funders do not focus their efforts on providing quality data, the public will remain confused, jaded, mistrustful and deprived of the information they need to make informed health and lifestyle choices.

Leave a Comment

Your email address will not be published. Required fields are marked *