Development and validation of a non-verbal consensus-based semantic memory paradigm in patients with epilepsy

 

Objective: Brain areas implicated in semantic memory can be damaged in patients with epilepsy (PWE) and neurodegenerative diseases; however, it is challenging to delineate semantic processing deficits from acoustic, linguistic, and other verbal aspects in existing clinical neuropsychological tests. We developed a new Visual-based Semantic Association Task (ViSAT) to overcome the limitations and systematically evaluate the extent non-verbal semantic processing dysfunction among PWE.

 

Background: The human brain can retain vast amounts of long-term general knowledge in the form of concepts, associations, raw facts, and other objective data. This cognitive domain is often referred to as semantic memory, and current neuroscience frameworks propose that its neural substrates are distributed throughout the brain in distinct cortical regions (“semantic hubs”) including in the temporal lobe. Patients with epilepsy (PWE), especially temporal lobe epilepsy, often have damaged circuits that overlap with some of these regions. However, whether PWE suffer from semantic processing deficits is an ongoing question because it is challenging to delineate impairments in semantic memory due to the reliance on language-based (verbal) tests. A non-verbal semantic processing task paradigm would help discern such dysfunction. The most well-known non-verbal semantic paradigm is the Pyramids and Palm Trees (PPT) task, in which participants are shown a single image at the top (“stimulus”) and two images at the bottom. They must select the one (“target”) image at the bottom that is most conceptually related to the stimulus. Performance is scored according to an “intended/correct” answer for each trial, but this could depend on individual context and life experiences. Thus, choosing a “less popular” answer would get scored as an incorrect response, despite successful semantic memory processing, with clinical implications (e.g., misdiagnosis) and implications for research (trial accuracy misclassification). We completely redesigned a Visual-based Semantic Association Task (ViSAT) to address these limitations in addition to other potential drawbacks by using online crowdsourcing approaches to obtain probability estimates of each answer choice. We validated this paradigm in control participants from the community (Controls) and PWE.

 

Method: The ViSAT was adapted from similar predecessors (Pyramids & Palm Trees test, PPT; Camels & Cactus Test, CCT) comprised of 100 unique trials using real-life color pictures that avoid demographic, cultural, and other potential confounds. We obtained performance data from 23 PWE participants and 24 control participants (Control), along with crowdsourced normative data from 54 Amazon Mechanical Turk (Mturk) workers. The ViSAT task was iteratively adapted to attain a high percent consensus (PCons) of the most popular answer choice for trials.

 

Results: ViSAT reaching a PCons >90% in 91.3% of trials compared to 83.6% in PPT and 82.9% in CCT. A deep neural network and image embedding model confirmed that correct answer choices were not related to the similarity of visual features (color, shape; i.e. non-semantic). The PWE group had lower accuracy than the Control group (p=0.019). PWE had longer response times than the Control group in general (p<0.001) and this was augmented for the semantic processing (trial answer) stage (p<0.001, mixed effects model).

 

Conclusions: The ViSAT task demonstrated impairments in PWE that may reflect dysfunction of non- verbal semantic memory circuits. This novel task is released open-source upon publication (currently under review) to facilitate non-verbal semantic memory research including future investigations of epilepsy and other conditions in clinical and research contexts.