ABSTRACT:

TITLE

 

Pilot Evaluation of a Custom GPT Model for Automated Quality Assurance of PET-CT Reporting

 

PURPOSE

To evaluate the feasibility of using a custom GPT-based large language model to support quality assurance (QA) in PET-CT reporting. The study assessed the model’s ability to highlight potential clinically significant errors and provide stylistic and structural refinements across a heterogeneous sample of real-world PET-CT reports.

 

METHODS AND MATERIALS

A bespoke GPT model was developed using a structured PET-CT QA prompt and two reference documents outlining reporting conventions and error definitions. 100 real PET-CT reports, authored by multiple specialist PET-CT radiologists, were exported in fully anonymised plain-text format (findings and conclusions only, no identifiers).

The model analysed each report for:

1. Potential clinically significant errors (laterality inconsistencies, anatomical mislabelling, numerical contradictions, or internal logical conflicts).
2. Non-clinical refinements (spelling, hyphenation, terminology standardisation, restructuring long sentences, and improving impression clarity).

Radiologists performed adjudication of only the AI-flagged potential errors; the entire dataset was not independently reviewed by humans.

 

RESULTS

The model flagged 5 potential clinically significant errors across the 100 reports. Subsequent human review determined that 4 of the 5 were false positives, each attributable to specific model or dictation limitations:

• One involved a correct anatomical description falsely flagged as inconsistent.
• Two were dictation artefacts (“lilac” → “iliac”; “colonostomy” → “colonoscopy”), judged unlikely to affect interpretation.
• One related to the model misinterpreting axial imaging orientation, incorrectly suggesting a laterality discrepancy.

One flagged case was confirmed to contain a genuine clinical error: a lung lobe laterality mismatch between the findings and conclusion.

 

Beyond error detection, the model generated 276 stylistic refinements (median 3 per report), addressing recurring typographical issues, harmonising terminology (e.g., “FDG-avid”, “ground-glass”), and improving clarity by restructuring overly complex sentences.

 

CONCLUSION

A custom GPT model can be safely applied to anonymised PET-CT reports to support QA by highlighting potential issues and enhancing clarity and consistency. While several AI-generated alerts reflected false positives, the tool demonstrated potential as a useful adjunct to radiologist oversight. Further prospective validation, including full human ground-truth review, is required before considering clinical deployment.