Poster Title: Quantifying Representation Bias In Alzheimer’s Disease Datasets: A Cross-dataset Risk Factor Analysis

 

Hello everyone. My name is Mansi Singhal, and I'm a Master's student in Biomedical Engineering at the University of Calgary. Today, I am presenting our work titled "Quantifying Representation Bias in Alzheimer's Disease Datasets: A Cross-dataset Risk Factor Analysis."

Artificial Intelligence has shown tremendous potential in Alzheimer's disease research, particularly for early diagnosis and disease progression prediction. However, AI models are only as good as the data they are trained on. Most existing models rely on a small number of commonly used datasets, such as ADNI, which may not adequately represent the diversity of the real-world population. This can lead to biased AI models that perform well for some groups but poorly for others.

To address this issue, our study introduces a novel bias score that quantifies representation bias across multiple Alzheimer's disease datasets.

We analyzed 10 major Alzheimer's datasets, including ADNI, OASIS, NACC, MIRIAD, BrainLat, DBPC, DLBS, HABS, PPMI, and SEA-AD. We evaluated 19 Alzheimer's disease risk factors, which we grouped into two categories: non-modifiable factors, such as age, sex, and genetics, and modifiable factors, including lifestyle, comorbidities, and environmental influences.

Our bias score ranges from −100% to +100%, where negative values indicate under-representation, positive values indicate over-representation, and zero represents balanced representation.

Looking at our results, we found substantial differences across datasets. For example, PPMI showed the highest level of under-representation with an average bias of −66%, while MIRIAD showed an average positive bias of +70%, indicating over-representation. Most datasets demonstrated considerable under-representation, with DLBS showing 100% under-representation, ADNI 83%, and DBPC 75%.

We also observed that both non-modifiable and modifiable risk factors were consistently under-represented across many datasets, and importantly, none of the datasets achieved balanced representation across all risk factors.

These findings are significant because biased datasets directly lead to biased AI models, reducing their fairness, generalizability, and clinical applicability. If we want AI systems that work reliably for diverse patient populations, we must first understand and address the biases present in the training data.

Our work provides a practical framework for quantifying representation bias before AI model development. We believe this can support the creation of fairness-aware AI models, improve dataset curation strategies, and guide future data collection efforts toward more representative Alzheimer's research.

Thank you.