Are the Amyloid and Tau Alzheimer’s Disease Literatures Equally Credible?

A Variable-Power P-Curve Analysis of Bias Signatures

Michael Flanders¹, Kendra D. Sims², Audrey R. Murchland², Isabel Elaine Allen³, Michelle Caunca³, Carlos Morales Valentín⁴, Lon S. Schneider⁵, Sarah F. Ackley¹

¹ Department of Epidemiology, Brown University, Providence, RI
² Department of Epidemiology, Boston University, Boston, MA
³ University of California, San Francisco, CA
⁴ Department of Psychology, University of Puerto Rico Río Piedras, San Juan, PR
⁵ Keck School of Medicine of USC, Los Angeles, CA

---

Alzheimer’s Disease Biomarker Research and Credibility

Alzheimer’s disease pathology. Amyloid-β plaques and tau tangles define the core pathology of Alzheimer’s disease (AD).

Credibility concerns. The amyloid field has faced disproportionate scrutiny related to retractions, data fabrication, overinvestment, and groupthink.

Reporting bias. Beyond individual cases of misconduct, we examine field-wide reporting patterns consistent with p-hacking and publication bias.

Aducanumab approval. The 2021 FDA approval of aducanumab provides a meaningful cut point for comparing earlier and more recent reporting patterns.

---

Research Question

Do the amyloid and tau Alzheimer’s disease literatures show distinct signatures of p-hacking and publication bias?

Do apparent differences narrow after the 2021 aducanumab approval, when the two literatures are compared within the same publication era?

---

Data Source and Comparator

PubMed Abstracts on Alzheimer’s Disease or Dementia

Source. PubMed-indexed abstracts from 1992 to the present that reference amyloid or tau alongside Alzheimer’s disease or dementia.

Comparator. Tau serves as an internal, field-matched comparison group. Amyloid and tau research share similar audiences, journals, and statistical reporting norms.

Why timing matters. Amyloid and tau show different publication trajectories over time. Because tau expanded later in the AD literature, calendar year is an important confounder when comparing their bias signatures.

---

LLM Extraction of P-Values From Abstracts

Pipeline. We used the OpenAI API and GPT-4o mini with zero-shot prompting to identify reported p-values in journal abstracts.

Validation. On human-checked samples, the extraction approach achieved approximately 90% precision and 90% recall. Larger models and few-shot prompting did not improve performance.

Why use LLMs? LLMs can capture the many natural-language forms in which p-values are reported, including variants that conventional regular-expression pipelines may miss.

---

P-Curve Methods

Variable-Power Gamma-Mixture Models

Not every study has the same probability of producing a small p-value. We model the observed p-value distribution as a mixture of studies with different levels of statistical power and estimate that mixture from the data.

Interval-Censored P-Values

Reported p-values are often intervals rather than exact measurements. Our models use the probability associated with each reported interval rather than treating values as exact points.

Examples:

• p = 0.04 → [0.035, 0.045)

• p < 0.05 → (0, 0.05)

Bias Signatures

Model-free cliff statistic.
Pr(p ∈ [0.04, 0.05)) − Pr(p ∈ [0.05, 0.06))

This statistic measures the discontinuity at the conventional significance threshold of p = 0.05.

Excess mass in (0.01, 0.05).
We fit models excluding the interval (0.01, 0.05), then compare the predicted versus observed p-value mass within that window.

---

Why P-Value Shape Matters

Under an unbiased literature generated by an honest normal model, p-curve densities should be:

• Smooth

• Non-increasing

• Convex on the interval (0, 1)

Any nonnegative mixture of these distributions, including a gamma mixture, preserves convexity.

Therefore, pronounced bumps or cliffs at p = 0.05 are inconsistent with an unbiased literature.

---

Results: Bias Signatures in Both Literatures

Both amyloid and tau show strong bias signatures:

• Sharp cliffs at p = 0.05

• Substantial excess mass in the interval (0.01, 0.05)

• More just-significant p-values than expected under the fitted honest-literature models

Simulation results show that honest p-value distributions do not produce cliffs of this magnitude.

---

Calendar Time Explains Much of the Apparent Amyloid–Tau Difference

Across the full 1992–present period, amyloid shows stronger bias signatures than tau:

• A larger discontinuity at p = 0.05

• Greater excess mass below p = 0.05

However, full-period comparisons combine an earlier, rapidly growing amyloid literature with a later-growing tau literature.

When analyses are restricted to the same recent publication period:

• Bias signatures remain large in both literatures.

• The amyloid–tau cliff difference is no longer clearly distinguishable from zero.

• The excess-mass gap becomes substantially smaller.

---

Interval-Smoothed Histograms

Reported p-values are imprecise intervals, not exact points. Rather than assigning each p-value to a single value, we distribute each observation’s probability mass across its reported interval.

Top: Uniform interval smoothing.
Amyloid (n = 7,223) and tau (n = 3,384) p-values from 1992 to the present were spread uniformly across their reported precision intervals before binning.

Bottom: P-curve gamma-mixture interval smoothing.
The same data were reweighted using a gamma p-curve mixture representing a moderate- to high-power honest literature.

In both approaches, the p = 0.05 cliff and excess mass below 0.05 remain, indicating that the observed bias signatures are not artifacts of interval smoothing.

---

Interpretation

Substantial p-hacking and/or publication bias signatures are present in both the amyloid and tau Alzheimer’s disease literatures.

Although amyloid appears more biased across the full study period, much of this difference is explained by calendar time, evolving reporting norms, and the different timing of expansion in the two research areas.

Comparisons of research credibility should therefore account for publication era rather than treating entire literatures as directly comparable across decades.

---

Funding

NIH National Institute on Aging: R00AG073454, P01AG082653, R35AG071916
NIH National Institute of Neurological Disorders and Stroke: UE5NS070680
Alzheimer’s Association: P0590624
Dana Foundation