PeptideStack
5.2kquestions
20kanswers
220users

Why do two papers on STEP 5 report different headline figures?

Asked 17 Oct 2025Modified 6 months agoViewed 11k times
15

The clinician who ordered the panel was not concerned; I would still like to understand it.

I keep seeing this stated as a fact with no explanation attached, and unexplained facts make me suspicious.

My background is quantitative but not chemical, so I can follow an equation more easily than a hand-wave.

What is the causal chain, and where does it stop being established?

clinical-trials
clinical-trials

Reading the primary literature properly: estimands, intention-to-treat versus per-protocol, confidence intervals, absolute versus relative…

745 questions
dosing-math
dosing-math

The arithmetic itself: milligrams to millilitres to insulin units, concentration after reconstitution, dose per draw, and vial-days per vial. Show…

764 questions
semaglutide
semaglutide

A GLP-1 receptor agonist with a fatty-acid-acylated backbone and a roughly one-week half-life, marketed for type 2 diabetes and for weight…

470 questions
shareeditfollowflag
DB
askedDr_Aoife_Brennan20k2717 Oct 2025

5 Answers

Sorted by votes
64

Because two papers on STEP 5 are usually reporting two different estimands from the same randomisation. The treatment-policy estimand asks what happened to everyone assigned, including those who stopped; the trial-product estimand asks what happens if you keep taking it. The second is always the larger number, and both are legitimate answers to different questions. Then there is the analysis population — randomised, treated, or completers — and the handling of missing data, where a last-observation-carried-forward and a multiple imputation can differ by a point or more. Neither paper is wrong. Read the statistical methods section and you will find both figures defined in it.

Look at the discontinuation rate alongside the efficacy figure. A large effect in the two thirds who stayed is a different result from a large effect in everyone.

A composite endpoint is only as informative as its least serious component. Where a cardiovascular composite combines death, infarction and stroke, ask which component moved, because they are not interchangeable outcomes.

Intention-to-treat and per-protocol analyses answer different questions. ITT asks what happens if you offer the treatment; per-protocol asks what happens if it is taken as directed. The gap between the two is a measure of how tolerable the protocol was.

Where a result is quoted from a conference abstract rather than a peer-reviewed publication, the numbers routinely move between the two. It is worth checking which one you are reading.

If a claim cannot be traced to a named trial with a named endpoint, treat it as a claim rather than as evidence.

shareimprove this answerflag
DK
answeredDr_Tomas_Kral53k3817 Nov 2025
Sponsored

Sigma-Aldrich - Certified Reference Materials

Analytical standards and reagents with traceable certificates. Every quantitative result you read inherits the accuracy of the standard behind it.

Shop standards
42

Before comparing two trials, check whether they share an endpoint definition. Frequently they do not, and the numbers then are not comparable in any sense.

Placebo arms in this class are not nothing. Lifestyle-intervention placebo arms in the major obesity trials commonly lose two to three per cent of body weight, so an active-arm figure quoted without its comparator overstates the drug effect by roughly that much.

Non-inferiority and superiority designs are not interchangeable. A non-inferiority result says the new agent is not meaningfully worse against a pre-specified margin — it does not say it is as good, and it certainly does not say it is better.

Meta-analyses in this area are dominated by whichever trial contributed the most participants, so read the forest plot rather than the summary estimate.

Quote the interval alongside the estimate and half the disagreements on this site would not start.

shareimprove this answerflag
DV
answeredDr_Bram_Verhoeven84k24828 Nov 2025
33

Start with what the trial was powered for. Everything else in the publication is secondary, exploratory, or a subgroup, and those three words mean three different things.

Trial populations are selected. Exclusion criteria in this class routinely remove people with significant renal impairment, prior pancreatitis and unstable psychiatric illness, which is exactly the population the results are then quoted for.

Mechanically, confidence intervals matter more than point estimates when two trials disagree. Two studies reporting fifteen and twenty per cent whose intervals overlap heavily have not disagreed about anything.

I am not a clinician and this is not medical advice; it is a reading of a published protocol.

When two sources disagree, the answer is almost always in the methods section of the one you have not read.

edited 14 Dec 2025 by h_pergande — fixed an arithmetic slip in the third paragraph

shareimprove this answerflag
HP
answeredh_pergande71k15810 Dec 2025
26

The honest answer here is that the published evidence supports part of the claim and is silent on the rest, and it is worth being precise about which part is which.

Open-label extensions are not the same evidence as the randomised phase. Once everyone knows what they are taking, the reported outcomes acquire a bias that no analysis fully removes.

Registry entries at ClinicalTrials.gov carry the pre-specified primary endpoint with a timestamp, which is the cheapest available check on whether an endpoint was changed after the data were seen.

One qualification: absence of a signal in a trial of this size is not evidence of absence for a rare event. It is evidence that the event is rarer than the trial could detect.

Read the protocol and the statistical analysis plan if the result matters to you. Both are usually published alongside.

edited 13 Jan 2026 by tyndall_haze — added the placebo-arm figures

shareimprove this answerflag
TH
answeredtyndall_haze38k3821 Dec 2025
6The placebo-arm figure is the part everyone omits. – Dr_Bram_Verhoeven 9 months ago
add a comment
20

More usefully, a trial establishes what happened to a defined group under a defined protocol. Extending it beyond that group is inference, and inference is allowed as long as it is labelled.

Duration decides what can be seen. A 68-week trial can measure weight and glycaemia; it cannot measure anything whose event rate is one per cent per year without enrolling tens of thousands.

The caveat is that trial evidence is about licensed product administered under supervision. None of it transfers automatically to research-grade material of unverified content.

The short version: check the endpoint, check the comparator, check who was excluded, then look at the number.

edited 29 Jan 2026 by Dr_Tomas_Kral — expanded the table to cover the lower concentration

shareimprove this answerflag
DK
answeredDr_Tomas_Kral53k381 Jan 2026
7Adding that the endpoint definition differs between the two trials being compared here. – tare_weight 8 months ago
6The exclusion criteria are the most informative page in the supplement and nobody reads them. – tyndall_haze 6 months ago
add a comment

Your answer

Ask PeptideStack is a static archive. Posting is closed, but the norms are worth stating: answer the question that was asked, show your working, cite the trial or the certificate, and say plainly where the evidence runs out.

Not medical advice. Research-use-only compounds are not approved for human use.