Title: Enrolling an ADRC clinical trial ready cohort: A machine learning prediction model

 

Author list:  Daniel Willie-Permor, MD, MPH1; Lilcelia A. Williams, PhD3,4; Sarah B. Berman, MD, PhD2,4 Beth E. Snitz, PhD2,4 Robert A. Sweet, MD2,3,4 Ann D. Cohen, PhD3,4 Oscar L. Lopez, MD2,4 , C. Elizabeth Shaaban, PhD, MS, MPH1,4,5

 

1Department of Epidemiology, School of Public Health, University of Pittsburgh    2Department of Neurology, University of Pittsburgh    3Department of Psychiatry, University of Pittsburgh 4University of Pittsburgh Alzheimer’s Disease Research Center 5Department of Health Promotion, School of Nursing, University of Pittsburgh

 

Background:
Numerous trials of pharmacologic and non-pharmacologic interventions are ongoing for Alzheimer’s disease and related dementias (ADRDs). These are most effective early in the disease process, before substantial neurodegeneration and cognitive impairment have occurred. For research centers seeking to enroll trial ready cohorts, a key question is how to target registry enrollment to maximize patients at this early end of the disease spectrum. We aimed to develop a prediction model to accurately identify such patients and allow more precisely targeted enrollment at an AD research center (ADRC).

Methods:
We analyzed application screening and diagnostic data from 192 participants evaluated at the University of Pittsburgh ADRC. Random forest classifiers were used to predict Clinical Dementia Rating (CDR) Global scores dichotomized as no impairment (CDR = 0) versus any impairment (CDR ≥ 0.5). Candidate predictors spanned domains commonly included in clinical research. Data was randomly divided into training (50%), validation (25%), and test (25%) sets. Classification trees were pruned using cross-validated misclassification error on the training set. Models were tuned using out-of-bag error and compared using performance in an independent validation set. Variable importance was examined to identify predictors contributing most strongly to classification.

Results:
Predictive performance depended on a limited subset of variables rather than the full candidate set. The selected random forest model demonstrated strong validation performance, with a validation accuracy of 0.872 (95% CI: 0.743-0.952) and balanced accuracy of 0.773. In the independent test set, the model achieved an accuracy of 0.813 (95% CI: 0.674-0.911) with balanced accuracy of 0.726. Variable importance (determined by the mean decrease in Gini, MDG) is shown in Fig. 1. Study partner-reported memory problems and participant-reported memory problems were the strongest contributors to classification, followed by age, educational attainment, MMSE, and living arrangement. Functional indicators, including difficulty managing money and medications, as well as study partner relationship and race, contributed additional but smaller discriminatory information.

Conclusions: Screening strategies focused primarily on participant or study partner cognitive complaints, living arrangements, and selected measures of functional difficulty, rather than by extensive additional screening data, could inform recruitment of trial ready cohorts aimed at early stage ADRD.