A clinician sees a pattern in their own patients. They run the numbers to check it, and the numbers seem to confirm what they already suspected. But that confirmation might not be real.
Adobe
“Often a clinician at the point of care will come to us with a hunch — ‘I’ve seen a bunch of these patients, I think this might be going on’ — and when you run the unmatched analysis, without statistical balancing, it can appear to confirm that hunch. It’s sort of the anecdote becoming the data point,” explained Brigham Hyde, co-founder of Atropos Health, a real-world evidence company whose platform generates observational studies from patient health record data on behalf of researchers and clinicians.
Once confounding variables are balanced through methods like patient matching, statistical analysis can reveal that a pattern does not hold up at the population level and is instead a reflection of a physician’s own patient mix or bias, he said.
As real-world evidence increasingly informs post-approval commitments, formulary decisions and label-expansion discussions, the difference between a matched and unmatched study could be the difference between a finding regulators can trust and one that reflects who happened to be included in the dataset.
Controlling for confounders
Propensity score matching, a statistical technique used in non-randomized observational studies to reduce confounding, matches patients who are similar to investigate the effect of a single variable.
“As with any real-world evidence study, nothing’s going to be a perfect match, but we use propensity score matching to align populations as closely as possible on clinical characteristics,” said Sun Kim, an endocrinologist and associate professor of medicine at Stanford University.
A study led by Jairo Noreña, then a Stanford endocrinology fellow, with Kim as senior author, found lower fracture incidence with semaglutide compared to sleeve gastrectomy, but Kim thought this could be due to higher weight loss with surgery.
“We thought a possible confounder was that sleeve gastrectomy patients had greater weight loss and therefore higher fracture risk, so we wanted to compare semaglutide to other weight-loss agents instead,” she said.
The new study, led by Velasquez JN, used Atropos Health’s real-world datasets to investigate the effects of semaglutide on bone fractures. The team evaluated changes in BMI and fracture incidence in people with type 2 diabetes taking semaglutide, dulaglutide or alternative weight loss therapies. The dataset covered approximately 60,000 patients.
Kim and her colleagues used propensity score matching to control for confounding variables such as age, gender, ethnicity and comorbidity score.
“You don’t want a fracture difference to show up simply because everyone in one arm happens to be young and healthy and everyone in the other arm is old and sick,” Hyde said.
To investigate bone fracture incidence compared to weight loss, Kim did a separate subgroup analysis using patients with BMI data recorded before and after treatment to check whether semaglutide patients had less fracture incidence even when they lost more weight.
The study found that semaglutide, despite delivering higher weight loss than other therapies, was not associated with increased risk of bone fractures. The findings, presented at the Endocrine Society’s ENDO 2026 meeting in June, showed that semaglutide use was linked to a 15% lower risk of bone fractures compared with alternative medications.
Why balance data matters
Controlling for confounding variables is only part of the story. “The other essential piece is transparency: you can see the balance tables, how confounding was evaluated, and use that to interpret and build confidence in the results,” Hyde said.
A balance table compares the matched groups on key characteristics such as age, gender and comorbidity burden, showing numerically how similar they ended up after matching. For sponsors and reviewers evaluating similar RWE claims, the questions Hyde and Kim describe — was matching used, were the matched groups actually balanced, is that data visible — are the ones worth asking before taking a topline finding at face value.
Observational research isn’t up to the gold standard of randomized controlled trials, but it is still an important tool, Hyde explained.
“The reality is we’re not going to run a trial for every single clinical question — it’s too expensive,” Hyde said. “Real-world data, handled properly with the right methodology and full transparency, is the best way to generate this kind of evidence at scale.”
Filed Under: Cardiovascular, Endocrinology
