Current approaches to language-based machine learning (ML) cognitive impairment detection often treat the task as an end-to-end binary classification problem, directly mapping speech or transcripts to diagnostic labels (e.g., dementia or normal cognition). While effective for accuracy, this binary framing collapses the progressive nature of cognitive decline and limits interpretability regarding which linguistic cues are clinically meaningful indicators of cognitive impairment. Recent advances using large language models (LLMs) further amplify this concern: LLMs can generate plausible-sounding outputs that are not reliably grounded in medically validated markers, risking misleading or fabricated cognitive assessments.
Rather than collapsing information into a single prediction, models- particularly LLMs- should be evaluated as cognitive analysis tools that extract interpretable signals (e.g., coherence, contextual information, and linguistic markers) prior to diagnostic inference. This direction requires systematic probing of LLMs, progression-aware modeling, and enriched dataset annotation strategies to align internal model signals with behavioral and linguistic cues clinicians can use.