Reading Health Research: A Short Guide for Non-Scientists

Health advice arrives daily, confidently, and often contradicting last week's. Coffee is good for you, then bad, then good again. Eggs, saturated fat, and intermittent fasting have all been through the same cycle.

The underlying research is usually less volatile than the coverage. What changes is which study gets reported and how. A handful of concepts is enough to tell whether a claim rests on strong evidence or on a small study with an eye-catching result.

Not all studies carry equal weight, and knowing roughly where a study sits tells you a lot before you read a word of it.

This is the most repeated principle in science communication and still the most violated. Observational studies can show that two things occur together; they cannot show that one causes the other.

The usual culprit is confounding — a third factor driving both. People who take vitamin supplements tend to be healthier, but they also tend to exercise more, smoke less and have higher incomes. Disentangling the supplement from everything that travels with it is genuinely difficult, and statistical adjustment is imperfect.

Reverse causation is the other trap. A study finding that people who eat less live shorter lives may have found that people who are ill eat less, not that eating less causes illness. Ask which direction the arrow points before accepting the headline's version.

This single distinction explains most alarming health headlines, and it is the fastest way to deflate them.

Suppose a study finds that a food increases the risk of a disease by 50%. That is a relative risk. If the baseline risk was 2 in 1,000, a 50% increase makes it 3 in 1,000 — an absolute increase of 0.1 percentage points. Both descriptions are accurate; only one is informative.

Whenever you see a percentage increase in risk, look for the absolute numbers. If they are not reported, that omission is itself informative.

A study of 20 people for four weeks tells you far less than a study of 2,000 people for four years, and both get reported with similar confidence.

Small studies are unstable — a couple of unusual participants can swing the result — and they are more likely to produce dramatic findings by chance, which is precisely why dramatic findings from small studies often fail to replicate.

Who was studied matters just as much. Findings from young male athletes may not transfer to post-menopausal women. Rodent studies are a starting point, not a conclusion — a great many interventions that work beautifully in mice do nothing in humans.

A p-value below 0.05 means the result would be unlikely if there were genuinely no effect. It says nothing about whether the effect is large enough to matter.

With a large enough sample, trivially small differences become statistically significant. A supplement that raises fat loss by 0.2 kg over six months can produce an impeccable p-value and still be irrelevant to anyone's life.

So ask about effect size, not just significance. And treat confidence intervals as more informative than p-values: an interval spanning from 'slightly harmful' to 'substantially beneficial' means the study has not settled the question.

Industry funding does not automatically invalidate research, but it is associated with results favourable to the funder often enough to warrant attention. Funding sources are usually disclosed at the end of a paper.

Publication bias is subtler and arguably more damaging. Studies with positive results are more likely to be published than those finding nothing, so the published literature systematically over-represents effects. This is a large part of why initially exciting findings tend to shrink as more evidence accumulates.

It is also why meta-analyses and systematic reviews sit at the top of the hierarchy: they attempt to account for exactly this.

You do not need to read the methods section to filter most claims. A few questions do most of the work.

A claim usually passes through several hands between the laboratory and your feed, and it changes at each step.

Preprints are papers posted before peer review. They are valuable for speed and entirely unvetted — a preprint is a manuscript, not a finding. Institutional press releases are the next link, and research has repeatedly found that exaggeration in news coverage frequently originates in the university's own press release rather than with the journalist.

By the time a study reaches a headline, causal language has often been added where the paper claimed only association, and the population has been generalised beyond who was studied.

The practical habit is to click through. If an article does not link the study, that is informative. If it does, the abstract alone usually reveals whether the headline is a fair summary — and often it takes under a minute to find out that it is not.

For most practical questions, individual studies are the wrong unit of information. Guidance from bodies that synthesise entire literatures — the WHO, national health services, professional medical associations — is more reliable than any single paper, and it updates when the evidence does.

PubMed abstracts are freely available if you want to check a claim's source, and the Cochrane Library specialises in systematic reviews written to be readable.

The most useful habit is simply this: when a claim seems dramatic, find what it is based on. Frequently the underlying paper is far more measured than the article describing it, which tells you most of what you need to know.