Background: Timely dementia identification remains a global challenge, impeding access to appropriate diagnostic and clinical care pathways for people living with dementia. There is growing interest in leveraging routinely collected Electronic Health Record (EHR) data for scalable dementia detection, but the quality and clinical utility of existing models remain unclear, with few progressing to implementation. This systematic review evaluated the performance, methodological quality, and risk of bias of EHR-based dementia prediction models.

Methods: We performed a systematic search of Medline, EMBASE, Scopus, IEEE Xplore, and ACM, supplemented by a comprehensive search of Google Scholar and preprint servers (arXiv, medRxiv, bioRxiv) to identify grey literature. We included studies that developed or validated probabilistic prediction models for identifying dementia (diagnostic models) or predicting future dementia (prognostic models) using EHR data. We synthesized results narratively and used PROBAST to assess risk of bias.

Results: Fifty-six studies describing 434 prediction models and 155 external validations were included. Most models were prognostic (66%), used US data (71%), relied solely on structured data, and 47 (11%) were externally validated. Modelled outcomes and prediction tasks were poorly defined in 63% of studies, making it difficult to determine if models aimed to identify diagnosed dementia, undiagnosed but existing dementia, or future dementia. For dementia ascertainment in model development, gold-standard clinical criteria were used in only 17 models (4%), with others reliant on diagnostic codes. Discriminative metrics were frequently reported (82% of models), but calibration rarely assessed (16%). Mean AUROC across all models was 0.76 for apparent performance, 0.79 for internal validation, and 0.74 for external validation. All models were judged as high risk of bias, driven by poor outcome definition, inadequate handling of missing data, and potential overfitting.

Conclusion: Our review identifies significant methodological issues in existing EHR-based dementia prediction models. Clinical usefulness and translation are likely limited by ambiguous outcome definitions, reliance on diagnostic codes for dementia ascertainment, and poor reporting. Clear definition of the model’s clinical intent, early interdisciplinary collaboration, and improved methodological and performance reporting may help advance clinical utility and realise the potential of EHR data for real-world dementia detection.