A contentious methodological debate has emerged within the neurology community, casting a shadow over how researchers quantify the link between brain plaque clearance and cognitive preservation in Alzheimer’s disease. A new analysis, published in JAMA Neurology, suggests that a popular statistical technique—known as quantile aggregation—may be artificially inflating the perceived strength of the relationship between amyloid-beta reduction and cognitive outcomes, potentially leading to overstated claims regarding the mechanisms of new-generation treatments.
The Core of the Controversy: Quantile Aggregation
At the heart of the critique is "quantile aggregation," a statistical approach that rearranges individual patient data into groups (quantiles) based on their post-treatment amyloid levels. By averaging the cognitive outcomes of all individuals within those specific groupings, researchers can visualize trends across a spectrum.
However, Sarah Ackley, PhD, of the Brown University School of Public Health, and her co-authors argue that this process masks the critical "noise" and variability inherent in human clinical trial data. By grouping participants, the method obscures the fact that individual cognitive decline is highly heterogeneous. When that variability is smoothed out, the resulting trends appear much cleaner and more robust than they truly are.
"In our simulations, aggregation inflated the strength of the apparent association 29-fold," Ackley and her colleagues reported. This suggests that what researchers and clinicians perceive as a definitive "dose-response" relationship—where higher levels of amyloid clearance equate to a predictable level of cognitive benefit—may, in some instances, be a statistical artifact rather than a clinical reality.
A Chronology of the Debate
The scrutiny of this method arrives on the heels of major shifts in the Alzheimer’s therapeutic landscape.
- 2023-2024: The pharmaceutical industry, particularly Eli Lilly, released secondary analyses from the TRAILBLAZER-ALZ 2 trial, which evaluated the monoclonal antibody donanemab (Kisunla). These analyses utilized quantile aggregation to demonstrate that patients who achieved deeper clearance of amyloid plaques showed a more pronounced slowing of clinical progression.
- Regulatory Milestones: Donanemab received FDA approval based on the primary findings of the TRAILBLAZER-ALZ 2 study, which demonstrated a statistically significant slowing of decline on standardized scales like the integrated Alzheimer’s Disease Rating Scale (iADRS).
- The Pushback: Following the release of the secondary analyses, independent researchers began to question the validity of the visual trends presented in the posters and publications. Critics argued that by combining placebo and treatment arms into the same quantile groups, the researchers were discarding the fundamental protection afforded by the original randomization.
- Current Findings: The JAMA Neurology research letter serves as the culmination of these concerns, providing simulated and real-world evidence that the aggregation technique produces near-perfect correlation values ($R^2$ values) that are mathematically unattainable when examining raw, individual-level data.
Supporting Data: When Statistics Collide with Reality
To test the reliability of quantile aggregation, the research team conducted a two-pronged validation study.
The Simulation Model
The authors constructed a simulated trial involving 1,600 participants, modeled after the physiological and cognitive profiles typically seen in early-stage Alzheimer’s populations. When they analyzed this data at the individual level—preserving the unique variance of each participant—they found a weak association between amyloid reduction and cognitive change, with an $R^2$ of 0.03. In statistics, an $R^2$ value close to zero suggests that the variable (amyloid) explains very little of the variance in the outcome (cognition).
When the team applied quantile aggregation to that exact same dataset, the $R^2$ jumped to 0.87. This transformation creates an impression of high predictive power where there is, in fact, almost none.
The A4 Study Evidence
The authors further analyzed data from the A4 study—a landmark Phase III trial of solanezumab that failed to show cognitive benefit. When individual patient data from the A4 study were analyzed, the correlation between amyloid levels and cognitive change was negligible ($R^2$ = 0.04). However, when the researchers processed this data through the quantile aggregation lens, the association became an almost perfect $R^2$ of 0.99.
This finding is perhaps the most damning: it demonstrates that quantile aggregation can make a drug that failed to provide any clinical benefit look as if it has a highly consistent, plaque-clearing "dose-response" effect on cognition.
Official Responses and Industry Defense
The implications of these findings are significant, but industry leaders maintain that the primary clinical outcomes of approved drugs remain untarnished by the debate over secondary analytical methods.
Mark Mintun, MD, Group Vice President at Eli Lilly, emphasized that the regulatory approval for donanemab rests on the primary endpoints of the TRAILBLAZER-ALZ 2 trial, not the exploratory secondary analyses.
"The results [of the primary trial] stand on their own, independent of any secondary analysis or methodological debate," Mintun told MedPage Today. He defended the use of quantile aggregation, framing it as a visualization tool rather than a definitive statistical proof. "The quantile aggregation approach was used as an exploratory tool to examine whether the degree of amyloid plaque clearance tracked with the degree of clinical benefit… We agree that quantile aggregation should not be interpreted in isolation."
Lilly’s stance is that clinicians and researchers understand that secondary analyses are intended to generate hypotheses—not to serve as the bedrock of efficacy claims. They contend that the visual clarity provided by these charts helps communicate complex data to a wider audience of physicians who may not be statisticians.
Scientific Implications: Moving Toward Causal Inference
The core of the issue, according to Ackley, is that the medical community must not conflate correlation with causation, especially when the methodology used to visualize that correlation violates the principles of randomized controlled trials (RCTs).
"When we do randomized controlled trials, randomization is the essential ingredient for causal interpretation," Ackley noted. "If we ignore this critical piece of information, we can’t conclude that amyloid reduction is responsible for cognitive benefit."
Proposed Methodological Shifts
The research team does not suggest that researchers stop investigating the relationship between amyloid and cognition. Instead, they propose a transition to more rigorous analytical frameworks that respect the sanctity of randomization:
- Causal Mediation Analysis: This method helps determine the extent to which the change in amyloid levels actually "mediates" or causes the change in cognitive outcomes, rather than just occurring alongside it.
- Instrumental Variable Techniques: A statistical approach used to estimate causal relationships when controlled experiments may have hidden variables or confounding factors.
- Within-Arm Analyses: By analyzing data within the treatment and placebo arms separately, researchers can better understand how the disease progresses naturally versus under the influence of the drug, without blurring the groups.
The Road Ahead: Transparency and Rigor
The debate underscores a broader tension in modern medicine: the need to make complex, often ambiguous clinical trial results accessible to the public and the medical community versus the need for strict, unvarnished statistical integrity.
As Alzheimer’s research moves into a new era of "disease-modifying" therapies, the burden of proof is increasingly high. Patients and families are looking for therapies that provide meaningful, long-term cognitive stabilization. If the pharmaceutical industry relies on statistical methods that may inadvertently "overstate" the benefits of amyloid clearance, it risks eroding the trust of the scientific community and the public.
For now, the consensus among methodologists is clear: visual tools that group patients by post-treatment outcomes should be viewed with significant skepticism. As the field looks toward future trials and new drug candidates, the emphasis must shift from "visualizing trends" to "verifying mechanisms." The use of more robust, transparent, and mathematically sound analytical techniques will be essential to ensure that when we claim a drug is working, we are seeing the truth—and not just a pattern shaped by the way the numbers were grouped.
