Background: Clinical and cognitive endpoints routinely deployed in Alzheimer’s disease (AD) clinical trials are vulnerable to administration and scoring errors. These errors can impact data reliability and validity, potentially obscuring treatment effects. Statistical approaches to data monitoring offer an objective approach to detect anomalous scores across large scale clinical trial datasets and identify opportunities for data quality improvement interventions. The aim of this work is to develop a data quality monitoring process, at scale, that can be applied to AD clinical trials.
Methods: Data were drawn from several multi-national AD clinical trials. Anomaly detection analyses were developed specifically for each study, configured for the set of scales and outcome scores included in the study. Analyses compared data from individual outcome scores within a scale, as well as combinations of outcome scores across one or multiple scales. These include analyses for statistically large changes, discordance between outcomes, and detection of multivariate outliers. Extreme, moderate, and mild outliers were identified across each analysis using distributional fitting and threshold cutoffs.
Results: A total of 41,245 visits with a combined 746,726 outcome scores across scales were examined. Analyses across all visits from all studies showed that 6% of visits contained at least one extreme outlier; 12% of visits contained at least one moderate outlier; and 24% contained at least one mild outlier. Extreme outliers were most common in analyses for discordance between outcomes measuring related constructs (8% of visits), followed by statistically large change in a single outcome (5% of visits), and multivariate outlier detection across outcomes (2.5% of visits).
Conclusion: Statistical data monitoring offers a scalable and efficient approach to identifying anomalous scores and potential errors in AD clinical trials. This approach creates opportunities for early detection of data quality issues that can be remedied by interventions, such as supplemental rater training and specific feedback on scale administration and scoring procedures. Statistical data monitoring can be integrated as an additional tool for maintaining data quality of cognitive endpoints in AD clinical trials.