Background

In cognitive aging research, synthesizing evidence across studies that use different cognitive measurements often requires outcome harmonization, for example by crosswalking effect estimates onto a common scale. A newly developed approach (cogxwalkr crosswalk) enables crosswalking of effect sizes based on different cognitive outcomes with minimal assumptions and has shown promise in prior work.¹ However, evidence on the performance of this approach remains limited; Comparing cogxwalkr against established item response theory (IRT) co-calibration methods would help validate the approach and clarify when it performs well.

 

Methods

To evaluate how well crosswalk and co-calibration approaches recover a target exposure–outcome association under realistic measurement conditions, we conducted a simulation study comparing cogxwalkr and co-calibration. We simulated two groups with different cognitive measures; the target estimand for the first group was the regression coefficient for education on cognition using the alternative cognitive measure from the second group. We examined four scenarios reflecting common harmonization challenges: (1) no differential item functioning (DIF) with strong anchors, (2) no DIF with weak anchors, (3) weak DIF with strong anchors, and (4) strong DIF with strong anchors. Education (30% college+) was generated as a binary exposure with a true effect of d = 0.2 or d = 0.4 on latent cognition. Outcomes were generated using item parameters estimated from the Alzheimer’s Disease Neuroimaging Initiative data. Co-calibration used item banking with anchor items to estimate harmonized scores, while cogxwalkr estimated a crosswalk factor in a third group using conditional random splitting and applied it to regression coefficients to recover the target effect.

 

Results

For each scenario, we compared the bias of both approaches in estimating this target education effect (Figure 1). Co-calibration showed greater variability across simulation replications (Figure 2). When compared to the target coefficient, both the cogxwalkr crosswalk and co-calibration demonstrated similar performance, with mean bias below 5% of the true effect size (Figure 3).

 

Conclusions

Crosswalk and co-calibration performed similarly,  suggesting that the cogxwalkr crosswalk provides a valid and practical solution to crosswalking effect sizes for evidence synthesis without requiring item-level data. Future research should consider how the cogxwalkr crosswalk performs until less idealized simulation conditions.

 

References

1. Ackley SF, et al. Methods to crosswalk between cognitive test scores using data from the Alzheimer's Disease Neuroimaging Cohort. Alzheimer's Dement. 2025;21:e14597.