The Application of Reverse Fragility Index to Plastic Surgery Randomized Controlled Trials with Statistically Nonsignificant Outcomes

Rodney Ahdoot, BS, Trista M. Benitez, MD, MPH, Chien-Wei Wang, MD, Kevin C. Chung, MD, MS

Section of Plastic Surgery, Department of Surgery, University of Michigan Medical School, Ann Arbor, MI

Background: Randomized controlled trials (RCTs) use p-values to assess significance, often relying on a threshold of 0.05. However, important study factors such as sample size, statistical power, and design quality may be overlooked. The fragility index (FI) measures how easily significant results can become nonsignificant, while the reverse fragility index (RFI) does the same for nonsignificant results. The reverse fragility quotient (RFQ) adjusts RFI for sample size. These metrics are particularly useful in plastic surgery, where small sample sizes and low event rates raise the risk of false-negative findings. In this study, we applied RFI and RFQ to plastic surgery RCTs with nonsignificant outcomes and explored factors associated with study robustness.

Methods: A systematic literature search of the 10 highest impact journals covering plastic and reconstructive surgery topics was conducted to identify RCTs published between 2000 to 2023. Eligibility criteria: (1) human participants, (2) used a 1:1 parallel arm study design, (3) published in English, and (4) contained at least one statistically nonsignificant primary dichotomous outcome mentioned in the abstract (p ≥ 0.05 or a 95% confidence interval containing the null value). The RFI was calculated by modifying the outcome events in a 2 x 2 contingency table. This was done by first identifying the group (e.g. intervention vs. control) with the fewer number of events, then iteratively subtracting events while simultaneously adding nonevents to this group, maintaining the number of participants to achieve a p-value less than 0.05 by Fisher’s exact 2-sided test. The total number of event reversals required to achieve significance was recorded as the RFI. The RFQ for each outcome was calculated by dividing the RFI by the sample size. Bivariate analysis was performed to investigate the association of trial characteristics with the RFI. 

Results: We screened 1,140 articles and included 40 RCTs with 65 statistically nonsignificant, dichotomous primary outcomes. Most studies were published in Plastic & Reconstructive Surgery (35%). Many trials lacked key methodological reporting: 30% did not clearly report loss to follow-up, 53% did not report a power analysis, and 35% noted potential underpowering. The median RFI was 4 (IQR 3–5); RFQ was 0.053 (IQR 0.038–0.08). The median sample size was 76. Of the 51 outcomes with clear loss-to-follow-up data, 31% had more loss than the corresponding RFI. Studies that acknowledged underpowering had significantly higher RFIs than those that did not (5 vs 4; p = 0.024). No other significant associations were found.

Conclusions: The findings of RCTs in the plastic surgery literature demonstrate fragility as the preservation of nonsignificant results rested on a small number of events. Incorporating the RFI into reporting standards can enhance the interpretation of study results by showing the number of event reversals that are needed to achieve significance. We recommend the routine reporting of the RFI and loss to follow-up alongside p-values, especially in studies with limited sample sizes.