Biomarker or Not: Automated Biomarker Detection in Alzheimer's Clinical Trial Text

Gaurab Subedi [1], Amanda Leisgang Osse [2,3], Andrew Hooyman [4], Yadi Zhou [5], Jefferson Kinney [2,3], Feixiong A. Cheng [5], Jeffrey Cummings, MD, ScD [2,3], Jorge Fonseca, Ph.D. [1]

[1] Department of Computer Science, University of Nevada, Las Vegas, Las Vegas, USA
[2] Kirk Kerkorian School of Medicine, University of Nevada, Las Vegas, Las Vegas, NV, USA
[3] Chambers-Grundy Center for Transformative Neuroscience, Las Vegas, NV, USA
[4] Physical Therapy, Crean College of Health and Behavioral Sciences, Chapman University, Orange, CA, USA
[5] Cleveland Clinic, Cleveland, OH, USA

Funded by the Alzheimer’s Drug Discovery Foundation (ADDF).

Introduction

Why this matters

This work is part of the larger ADBMO effort to accelerate biomarker annotation in Alzheimer’s disease clinical trials.

Clinical trial records contain biomarker-related eligibility criteria and outcome measures, but manual review is slow and expert-intensive.

We automatically identify biomarker presence and absence in trial text, narrowing expert review to only the passages that require annotation.

Goal

Benchmark AI models for biomarker mention detection to support faster trial triage and downstream expert annotation.

Methods

Data Source

ClinicalTrials.gov Alzheimer’s disease trials

Dataset

1,900 expert-annotated lines from 234 trials

Data Splitting

Stratified splits by NCTID to prevent data leakage:

Train: 70%
Validation: 15%
Test: 15%

Feature Conditions

TF-IDF
TF-IDF + BioWordVec-200
TF-IDF + FastText-300

Classical Models

Logistic Regression
Support Vector Machine
Random Forest
XGBoost
Decision Tree
Artificial Neural Networks

Transformer Models

Fine-tuned transformer models using AdamW and class-weighted loss:

BERT-base
BioBERT
PubMedBERT
Bio+ClinicalBERT
BioLinkBERT

Zero-shot transformer models with no fine-tuning:

BART
FLAN-T5
BioGPT

Evaluation

Five-fold cross-validation stratified by NCTID was used.

Results

PubMedBERT was the strongest single model, achieving CV F1 = 0.9660 and AUC = 0.9947.

The soft-vote BERT ensemble achieved CV F1 = 0.9702 and perfect AUC = 1.000 on the held-out test set.

Logistic Regression reached CV F1 = 0.9469, while SVM with BioWordVec-200 embeddings, pre-trained on PubMed and MIMIC-III, reached CV F1 = 0.9658.

Zero-shot models performed poorly. FLAN-T5 was the best zero-shot model, reaching F1 = 0.6283.

On the test set, the four leading BERT models and the classical ensemble converged at weighted F1 = 0.985, with biomarker-class F1 = 0.977.

Speed for Trial Triage

Classical SVM with BioWordVec-200: approximately 50 ms
BERT-based models: approximately 2–4 seconds

Discussion

Strong Classical Alternative

SVM with BioWordVec-200 nearly matched the best transformer model while offering much faster inference, making it practical for large-scale trial triage.

Efficiency Matters

Fast inference enables scalable screening of trial records before expert annotation.

High Accuracy

Transformer models, especially PubMedBERT, achieved the best performance on both cross-validation and the test set.

Zero-Shot Limitations

General foundation models without task-specific training performed poorly.

Conclusion

An SVM trained using BioWordVec embeddings produced results comparable to transformer models but with faster inference. This shows that carefully trained classical models with quality data can still be a competitive and efficient alternative to transformer models.

Task-specific training improved identification of biomarker-relevant text within the ADBMO pipeline.

Automated biomarker flagging reduced annotation workload by directing experts to relevant text and filtering out filler.

Contact

Gaurab Subedi