•Measuring quality of care, represented by quality indicators (QIs) is a key imperative for healthcare systems globally, including for residents of aged care.

•To do this at scale and efficiently, algorithms can be used to automatically extract QIs from care records.

•However, algorithms rely on data being collected in structured fields, rather than free-text fields. If information in care records is documented in free-text data fields, QIs extraction relies on manual record reviews which is time-consuming and costly.

Objective:

To analyse the proportion of QIs that are documented in residents’ care records in free-text or structured fields. This is a sub-study of the Australian CareTrack Aged study, which is the first systems-level study in the world of evidence-based care delivered to residents in aged care facilities (1).

Methods

The CareTrack Aged study developed 233 QIs which represent evidence-based care for residents of aged care facilities across 16 conditions, including Cognitive impairment and Medication management (1,2) (Figure 1). The QIs were developed by systematic searches of peer-reviewed literature and review by national experts (researchers and clinicians) via a Delphi process(1, 2).

Trained aged care nurses undertook manual care record reviews to identify the data field types used (structured, free-text, hybrid i.e., combined free-text and structured) to determine eligibility and adherence with evidence-based QIs for care delivered to residents in 2021(1). See Box 1.

Results 

Data field type was extracted from 6 electronic care record systems across assessments of QI eligibility (n=8343) and compliance (n= 4935) for 114 residents in 40 facilities. Most frequently, QI eligibility relied on free-text fields (44% of assessments, n=3642) and QI adherence on a hybrid of free-text and structured fields (60%, n=2950) (Figure 1). Using only structured data, 17% (n=1425) of QI eligibility assessments were possible, and 5% for adherence (n=251). None of the 233 QIs were assessable for eligibility and adherence using only structured data.

Conclusion

In this study, QI assessments of aged care residents’ records relied mostly on free-text and hybrid data fields, meaning QI extraction algorithms would not be currently usable at scale. These findings can be used by aged care researchers and accreditors to inform their data extraction approaches and provides a basis for the future design of QIs and care record systems to leverage structured data fields and automation. In this way, scalable and replicable care quality assessments can be made possible for benchmarking purposes. 

Artificial intelligence tools such as natural language processing may offer improved methods to automatically extract QI data.

References:

1.  Hibbert PD, Wiles LK, Cameron ID, Kitson A, Reed RL, Georgiou A, Gray L, Westbrook J, Augustsson H, Molloy CJ, Arnolda G, Ting HP, Mitchell R, Rapport F, Gordon SJ, Runciman WB, Braithwaite J. CareTrack Aged: the appropriateness of care delivered to Australians living in residential aged care facilities: a study protocol. BMJ Open. 2019;9(6):e030988.

2. Hibbert PD, Molloy CJ, Wiles LK, Cameron ID, Gray LC, Reed RL, Kitson A, Georgiou A, Gordon SJ, Westbrook J, Arnolda G, Mitchell RJ, Rapport F, Estabrooks C, Alexander GL, Vincent C, Edwards A, Carson-Stevens A, Wagner C, McCormack B, Braithwaite J. Designing clinical indicators for common residential aged care conditions and processes of care: the CareTrack Aged development and validation study. Int J Qual Health Care. 2022;34(2).