Health Study Hype

The media are trusted to decipher complicated scientific studies and deliver the straight goods to the public. Unfortunately, their take on the big health news of the day sometimes sacrifices accuracy for sensationalism.

How does this happen? Abstracts – the brief summaries of pages-long research papers – often exaggerate the positive outcomes of the study, according to a cohort study published in PLOS Clinical Trials. Forty percent of the abstracts examined by researchers spun the results more positively than they actually were, and 47 percent of the press releases associated with the research papers studied had a similar bias. The researchers found that this caused a domino effect: because of the angles of the abstracts and press releases, more than half of the news items reporting on these studies also echoed this “spin.”

Lee Falin, PhD, host of The Everyday Einstein’s Quick and Dirty Tips podcast (quickanddirtytips.com) doesn’t lay the brunt of the blame on the abstracts themselves, though. “I think the problem is a failure to understand the limitations and caveats associated with the research,” he notes. “Part of that is because those limitations usually aren’t mentioned in the abstract, and most people don’t have full access to the study.” (The full text of many studies is often available on a subscription-only basis or hidden behind a pay wall.)

The bigger problem, he says, is that most people – the media included – don’t understand how to look at the methods used and deduce whether or not the results are significant. Other issues that the regular reader may not recognize include confounding factors that the researchers may not have accounted for, and whether the conclusions drawn have done a good job of separating correlation and causation. (For example, if a study found that ninety out of a hundred people who lost 20 pounds ate carrots every day for a year, that really isn’t enough proof that the carrots were the cause of the weight loss.)

Something else to be aware of? The company or organization that paid to help make the study possible. “Unfortunately, funding information is available in the abstract only about half of the time,” Falin points out. “I think that individuals, especially policy makers, need to know who is funding the research they are basing their decisions on.”

Overall, Falin emphasizes that you have to be aware of the study’s design and its limitations. After all, as he says, “For every study you find claiming one thing about a certain food, you can find another making the exact opposite claim.”

Here are Falin’s takes on two recent headlines that prove you can’t always take a headline at face value.

“Eggs are worse for you than smoking!”

This was in reference to a study conducted on more than 1,200 people, in which the researchers concluded that eating egg yolks spurs the formation of plaque buildup in the arteries more so than smoking does. However, Falin points out that there were details that the news coverage did not address (for example, that the average age of the participants was 62 and that they were patients at a vascular prevention clinic, facts that indicate that their health may not have been so good in the first place); additionally, there were many points that the researchers didn’t take into consideration, such as how the eggs were prepared or if there were other foods eaten alongside the eggs. “These are all confounding factors that the study doesn’t account for, but you can’t tell that from the abstract alone, nor from the news article,” he says.

“Organic food isn’t any better for you!”

Recently, blogs were abuzz over a study that concluded there are really no extra nutritional benefits in choosing organic over conventionally grown food. However, Falin notes that some of the articles covering this research strongly emphasized one half of the story, but deemphasized the other half. He notes that the original study concluded: “The published literature lacks strong evidence that organic foods are significantly more nutritious than conventional foods. Consumption of organic foods may reduce exposures to pesticide residues and antibiotic-resistant bacteria.”

That means that while organic foods may not contain more vitamins and minerals, eating them may be a way to avoid harmful pesticides. Unfortunately, the second, positive point was often omitted or buried within media coverage.