TITLE: Diagnostic cohort study comparing the accuracy of PDPH Trial Criteria vs International Headache Society Criteria for post-epidural PDPH

AUTHORS: Angle P1, Lay C2, Gawel M1, Pardy B3, Kiss A4, Silva M1, Kong J1, van DerVyver M1, Roberts 1L, Hussain K1, Halpern S1.

1 Department of Anesthesia, University of Toronto, Sunnybrook Health Sciences Centre

2 University of Toronto, Headache specialist/Neurology, Women’s College Hospital

3 OARU, Sunnybrook Health Sciences Centre

4 University of Toronto, Department of Research Design and Biostatistics, Sunnybrook Health Sciences Centre

Original members, Canadian Multicentre PDPH Trial: Angle P, Kronberg J, Halpern S (Women’s College Hospital),  George R, McKeen D (IWK hospital, Halifax) Douglas J, Preston R (BC Women’s Hospital), Morely-Forster P, Singh I, Dhir S (London Health Sciences Ctr), Christine Lay (Women’s College Hospital), Marek Gawel.

INTRODUCTION

In obstetric anesthesia practice, epidural needles, rather than spinal needles, represent the most important source of post dural puncture headache (PE-PDPH). 

Despite the important differences that exist between epidural and spinal procedures and their likely impact on PE-PDPH, these differences have yet to be meaningfully addressed by  International Headache Society (IHS) criteria for PDPH diagnosis (see Figure 1).

They include:

1)  high incidence of unrecognized dural sac injury during epidural needle placement;

2)  variability in dural injury during accidental dural puncture which may effect PE-PDPH onset and severity;

3)  potential diagnostic confounding by pneumocephalus from LOR to air techniques; and,

4) continued omission of postural neckache without headache as a PDPH variant despite literature which supports its inclusion.   

Diagnostic criteria for PE-PDPH were developed to address these issues and used as the primary outcome in the Canadian Multicentre PDPH Trial (CMPT; PDPH Trial Criteria, Fig. 1). This Trial compared the incidence of PE-PDPH within the first 14 days of labor epidural placement in 1081 women randomized to receive small (19g) vs large (16-18g) Tuohy-type needles. All women were followed forward for possible PE-PDPH over 14 days after epidural placement as measured by a Triage Test (i.e. development of a “bothersome headache or neckache that worsened with movement of any kind” during standardized follow). PE-PDPH was diagnosed using PDPH Trial Criteria (primary outcome) and IHS ICHD-11 Criteria (secondary outcome) by a blinded independent adjudicating body consisting of three senior anesthesiologists matched 1:1 with an expert headache specialist neurologist for diagnosis.

An embedded diagnostic study in the Trial permitted comparison of the accuracy of PDPH Trial Criteria with IHS ICHD-11 Criteria < 14 days after epidural placement against an Umpire Reference Test (an independent, blinded diagnosis of symptoms consistent with CSF leakage made by a second expert headache specialist/neurologist based on standardized long-term patient follow up (to symptom resolution, maximum 1 year) and clinical experience). Trial findings suggested that the diagnostic performance of PDPH Trial Criteria was superior to IHS ICHD-11 Criteria for PE-PDPH (see Figure 2). 

IHS ICHD-3 Criteria [1] were introduced shortly after conclusion of the CMPT and are now advocated for diagnosis of PDPH. This retrospective cohort study used case report forms from women identified with possible PE-PDPH in the Trial to compare the diagnostic accuracy of PDPH Trial Criteria versus IHS ICHD-3 Criteria against the same Umpire Reference Test.

Primary Research Question: What is the performance of PDPH Trial Criteria versus IHS ICHD-3 Criteria for PE-PDPH diagnosis within the first 14 days of labor epidural placement when assessed against the Umpire Reference Test (long-term follow up/expert opinion by a headache specialist/neurologist) used in the CMPT?

Secondary Research Questions: How do PDPH Trial Criteria compare with IHS ICHD-11 Criteria for PE-PDPH diagnosis based on the Umpire Reference Test? How do diagnoses based on clinical experience of each adjudicator compare with the Umpire Reference Test? How do adjudicators’ recommendations for Epidural Blood Patch (EBP) in true positive cases of PE-PDPH compared within 14 days of epidural placement?   

METHODS

After REB approval, case report forms for 184 women identified with a positive triage test for possible PE-PDPH within 14 days of epidural placement in the CMPT were assessed using three Index Tests and based on the clinical experience of Day 14 adjudicators (Fig. 3 Study Flow Diagram, Fig. 4 Demographic Table n=184).

PE-PDPH diagnoses were made by six mid-career to senior staff obstetric/regional Anesthesiologists randomly allocated to adjudicate each case in fixed pairs using either PDPH Trial Criteria, IHS ICHD-11 criteria, or IHS ICHD-3 Criteria. During evaluations, adjudicators first independently assessed each case for PE-PDPH using their assigned diagnostic criteria. Adjudicators were blinded to diagnoses made by their adjudicating partner and other adjudicators, as well as the needle used during epidural placement. First pass agreement for PE-PDPH was assessed between adjudicators for each set of diagnostic criteria (kappa statistic). When discordance was found, the case was re-reviewed and a consensus diagnosis of PE-PDPH (yes/no) was made.     

 

The same anesthesiologist assessors, as well as a second independent, blinded headache specialist/neurologist, also rated each case for PE-PDPH (yes/no) within the first 14 days of epidural insertion based on their clinical experience. Assessors identified cases they believed should have been recommended/received an EBP (regardless of what actually happened in the Trial), as well as cases they believed warranted >14 days of follow up and/or medical imaging as part of PE-PDPH diagnosis.  

 

Diagnostic performance of each Index test was assessed against an Umpire Reference Test using independent, blinded assessments made by a second expert headache specialist/neurologist who assessed each case for symptoms consistent with CSF leakage based on long-term follow up (symptom resolution, maximum 1 year) from the Trial.

 

Measures of Diagnostic Performance were:

 

      a) Sensitivity

      b) Specificity

      c) Area under the Curve (AUC) for Receiver

          Operating Characteristic (ROC) curves [2]

      d) Likelihood Ratios (LR)

      e) Positive and negative predictive values

 

RESULTS

Thirty-one out of 1081 women (2.9%) women were diagnosed with PE-PDPH by the Umpire Reference Test. Characteristics of PE-PDPH cases are provided in Figure 5.  

 

Diagnostic performance of each set of criteria compared against the Umpire Reference Test are provided in Figure 6.  

 

• First pass agreement between adjudicators was substantial for PDPH Trial Criteria (kappa 0.62 p < .001) vs fair for IHS ICHD-11(kappa 0.26, p < .001) and IHS ICHD-3 (kappa =0.26, p <.001).

 

• PDPH Trial Criteria were more sensitive for PE-PDPH (48%, 15/31) compared to IHS ICHD-11 (13%, 4/31) and IHS ICHD-3 (32%, 10/31); all three diagnostic criteria showed high specificity (98-99%).

 

• Overall diagnostic accuracy for PDPH Trial Criteria was good to acceptable (AUC 0.73, 95% CI [0.64, 0.82]), unacceptable for IHS ICHD-11(AUC 0.56, 95% CI [0.44, 0.68]) and low for IHS ICHD-3 Criteria (AUC 0.66, 95% CI [0.57, 0.74]).

 

Independent agreement between adjudicators’ recommendations for EBP in “true positive” cases (based on adjudicator consensus and the Umpire Reference Test) varied considerably depending on the diagnostic criteria used (see Figure 7).

Adjudicators using PDPH Trial Criteria identified two cases (14%, 2/14) they believed required >14 days follow up and one case (7%, 1/15) for medical imaging while IHS Criteria identified no cases for either. The neurologist assessing diagnoses within 14 days of epidural placement identified one case (4.8%, 1/21) which required follow up >14 days and 11 cases (50%, 11/22) where they believed medical imaging was warranted.  

The lower sensitivity of IHS Criteria ICHD-11 and ICHD-3 resulted in a large proportion of missed cases of PE-PDPH with ICHD-11 missing 27 cases (87%, 27/31) and ICHD-3 missing 21 cases (68%, 21/31) within the first 14 days of epidural placement, based on the Umpire Reference Test. 50% of the missed cases reported symptoms beyond 14 days after epidural insertion with average pain ratings of 6-7/10.  

Over half of cases of PE-PDPH that were missed by all diagnostic criteria 56% (15/27) reported bothersome PDPH-related symptoms beyond 14 days; about 1/3 (33%, 8/27) of women reported having PDPH-related symptoms up to 13 weeks post epidural.  

 

DISCUSSION

PDPH Trial Criteria demonstrated overall superior diagnostic performance for PE-PDPH (AUC 0.73, 95% CI [0.64, 0.82]) compared with IHS ICHD-3 (AUC 0.66, 95% CI [0.57, 0.74]) and IHS ICHD-11 Criteria (AUC 0.56, 95% [CI 0.44, 0.68]) during the first 14 days after labour epidural insertion compared with the Umpire Reference Test. This is consistent with findings from the embedded diagnostic study in the CMPT which compared IHCD-11 Criteria versus the PDPH Trial Criteria using very senior, anesthesiologist expert adjudicators.

Similarly, both studies showed that interrater agreement was better for PDPH Trial Criteria when compared with either set of IHS Criteria.

PDPH Trial Criteria also demonstrated higher sensitivity than both IHS ICHD-3 and ICHD-11 Criteria, with similarly high specificity. These findings are consistent with our original comparison of the diagnostic accuracy of PDPH Trial Criteria with IHS ICHD-11 criteria for PE-PDPH against the same Umpire Reference Test in the CMPT.

Although IHS ICHD-3 criteria clearly outperformed IHS ICHD-11 Criteria, PDPH Trial Criteria showed overall better performance suggesting that these criteria are preferred as part of development of a new diagnostic pathway for PE-PDPH.

Long-term follow up of cases identified by the Triage Test showed that a large number of women developed PE-PDPH beyond 5 days after epidural insertion suggesting that a diagnostic window of 14 days is preferable. Similarly, many women continued to experience PE-PDPH related symptoms longer than 14 days after epidural insertion suggesting longer term follow up of these women is required.  

Lastly, little consensus was found between the headache specialist/neurologist and the anesthesiologist adjudicators for EBP recommendations for true positive cases of PE-PDPH based on the Umpire Reference Test. This suggests that an individualized approach to clinical care is being used and that further research is required.

Funded by: The LANGAR FOUNDATION, Toronto, Canada