Dr Alexander Lomas - Royal Cornwall Hospital, Dr Malcolm Broom - Glasgow Royal Infirmary/Princess Royal Maternity

Minority ethnic groups in the UK have poorer perinatal outcomes compared to equivalent white parturient groups (1).

Language is a contributor to this.

Machine translation tools exist, but to date have been inaccurate and unsuitable for medical communication (2). Despite this, they are regularly used in healthcare.

Large Language Models are rapidly developing and capable of translation tasks, but can they be used for patient communication?

Methods

GPT-4 (3) was used to translate both a patient information leaflet on epidurals and generate it’s own patient information from a reference website.

Five languages selected based on worldwide prevalence: Bengali, Hindi, Portuguese, Spanish, Mandarin.

Freelance interpreters asked to grade the translations using a three-domain Likert Scale (4) and to translate the version created by GPT-4.

Results Information translated by GPT-4 was generally accurate, with little to no misleading information or omissions.

Global rating of quality was high. There were some grammatical errors which affected readability but the text remained understandable.

Information generated by GPT-4 from a reference was overall more fluent and readable. It contained appropriate information on epidurals and risks.

Reverse translating with the model was an accurate way of determining the content.

Interpreters’ Comments

“The translation is perfect. It reads smoothly and sounds natural in Hindi.”

“A high level of professionalism, reflecting an understanding of medical terminology and the ability to convey complex medical procedures in a clear and accurate manner.”

“Balance between technical accuracy and readability is commendable.”