How to read medical studies: a patient’s guide
Every time you see your doctor or browse the internet, you hear phrases like: “studies show this treatment is more effective”, “new research proves…”, “the evidence supports…”. But how do you tell a good study from a weak one — and why does it matter to you?
The quality of a study determines whether the treatment being offered to you rests on solid proof or on guesswork. Below are the five most important criteria — in plain language.
Randomisation: the only fair way to compare
Randomisation means that patients in a study are allocated by chance — like the toss of a coin — to one or the other of the treatments being compared. It is the only fair way to compare two treatments.
A study without randomisation is almost always flawed. Why? Because the doctor may unconsciously give the new treatment to patients with better prospects — and then the new treatment seems to “work better” when in reality nothing has changed.
What to ask: “Was the study randomised?”
What the study measures — and what it should
Many IVF studies focus on intermediate numbers: how many eggs were retrieved, how many embryos created, how many had normal chromosomes. These are interesting — but what matters to you is not the number of eggs. It’s the child in your arms.
A well-designed study measures the final outcome: the birth of a healthy child (live birth rate). When you read “the new method increased egg numbers by 20%”, the right question is: did it also increase the number of babies?
What to ask: “What is the live birth rate?”
Statistically significant does not mean significant for you
This is where most confusion arises. When a study says a result is “statistically significant”, it only means that the difference found is probably not due to chance. It does not mean the difference matters to your life.
Example: A study with thousands of patients shows that a new treatment increases egg numbers by 1 (from 10 to 11). Statistically significant? Yes. Meaningful for you? No.
Conversely: a small study may show that a treatment improves pregnancy rates by 5% — but the result is not “statistically significant” because the study was too small to prove it. That does not mean the treatment does not work — it means we need larger studies.
Study size — and why meta-analyses matter
A study with 30 patients may make headlines — but its conclusions are uncertain. Large studies with hundreds or thousands of patients (often from many centres) are far more reliable.
Even better: meta-analyses — studies that pool data from many previous studies to give the overall picture. When you read “a meta-analysis shows…”, that is usually stronger evidence than a single study.
A meta-analysis, however, is not a panacea. Its value depends on the quality of the studies it combines: a meta-analysis of randomised trials is far stronger than a meta-analysis of retrospective studies. When the source data are weak, the meta-analysis does not make them stronger — it merely summarises them.
The strongest evidence of all: when international guidelines (ESHRE, ASRM, NICE) recommend a treatment, it means panels of experts have reviewed all the available evidence and reached consensus.
Ideal conditions vs everyday practice
There are two distinct types of studies:
Efficacy studies ask whether a treatment works under ideal conditions — selected patients, strict protocols, specialist centres. They answer: “Can this treatment work?”
Effectiveness studies ask whether the treatment works in real life — across the full diversity of patients. They answer: “Does it actually help in everyday clinical practice?”
Both matter. For your personal decision, however, the more relevant answer is the second — because you are not being treated under ideal conditions; you are being treated in real life.
Summary: what to remember
When your doctor recommends a treatment because “the studies show it”, the right questions are:
- Was the study randomised?
- What was the final outcome — birth of a child, or just an intermediate number?
- Is the difference clinically meaningful, not only statistically significant?
- How many patients were studied?
- Is there a meta-analysis or an international guideline that supports it?
You don’t need to become a statistician. You just need to know which questions to ask.
