1 Introduction

Distal radius fractures (DRFs) are the most common upper extremity fractures in the United States, with an estimated incidence of 33 % of all fractures in the pediatric population and 18 % in the elderly.^1–4 Improper management of DRFs can lead to severe complications such as wrist stiffness, chronic pain, severe arthritis, decreased grip strength, malunion, and nerve injury.⁵

While closed reduction and immobilization is a non-surgical treatment option for DRFs, surgical interventions are widely used and include volar locking plates (VLPs), percutaneous pinning using K-Wires, intramedullary nailing, and others.⁶ Among these surgical options, VLP is the gold-standard treatment choice as it leads to early functional recovery and favorable clinical outcomes.^6,7 When compared against casting, multiple level I evidence studies demonstrate VLP superiority as well. For example, Stephens et al. demonstrated significantly better DASH scores for VLP than casting both at 3 and 24 months in a meta-analysis of randomized controlled trials (RCTs).⁸ However, despite level I evidence demonstrating the clinical superiority of VLPs, many surgeons continue to choose casting based on factors such as cost-effectiveness and personal preference.^9–11 The persistent use of casting instead of VLP despite strong level I evidence in favor of VLPs poses questions about their ostensibly clear superiority, which might have a weaker scientific foundation than reported in the literature.¹² Some existing research has pointed to this discrepancy as well, such as a systematic review and meta-analysis by Franceschi et al., which found no superiority of VLP over K-Wire fixation in terms of range of motion, grip strength, radiographic outcomes, or total complication rates.¹³

The concept of statistical fragility is particularly relevant for interpreting RCTs, which represent the highest level of evidence in clinical research. While the p-value is often used as the gold-standard measure of significance, it is often criticized for failing to consider study design elements and the number of patients lost to follow-up.^14,15 Feinstein therefore introduced the concept of the fragility index (FI) as a supplement to the p-value to address its limitations.¹⁶ The FI represents the statistical fragility of an outcome. It is calculated as the number of iterative outcome event reversals needed to lose statistical significance, and has been widely reported in orthopaedic surgery literature.^17–20 The reverse fragility index (rFI) is similarly defined to represent the number of outcome event reversals required to turn non-significant outcomes into statistically significant findings.^21–23 The fragility quotient (FQ) was introduced to take sample size into consideration and is determined by dividing FI by sample size.^24,25 Thus, the FQ represents the proportion of patients that need an outcome event reversal for significance to change for a certain outcome.

Although numerous level I evidence studies demonstrate that distal radius fractures have the most favorable outcomes when treated with VLPs, alternative options, such as casting, seem to persist in the modern hand surgeon's armamentarium. The dissonance between surgeon experience and level I evidence may partially stem from a weak statistical foundation of RCTs, driven by statistically fragile outcomes. This study therefore sought to investigate the statistical fragility of RCTs comparing volar plating and casting for the treatment of distal radius fractures.

2 Materials and methods

2.3

2.3 Fragility analysis

To confirm the reported p-values, a two-tailed Fisher's exact test was used, and statistical significance was set at a p-value <0.05. Outcome events were then adjusted until the minimum number of event reversals causing significance reversal was found (Fig. 1). This value, either the FI or rFI depending on the significance of the original outcome, was then recorded. Finally, the FQ was calculated for all outcomes by dividing the FI or rFI by the study sample size. All fragility statistics (FI, rFI, and FQ) were reported as medians with a corresponding interquartile range (IQR), both for all outcomes extracted and individual subsets based on outcome significance, defined outcome categories, year of publication, and lost to follow up status.

Close

Fig. 1 Demonstration of statistical significance reversal using a 2 x 2 contingency table with a resulting fragility index (FI) = 1 (Martinez-Mendez et al., 2017).

Export to PPT

3 Results

699 RCTs were identified using the initial search terms, of which 120 met full text screening criteria. 13 of those studies were included for final analysis (Table 1, Fig. 2). A total of 36 dichotomous outcomes were extracted, with a median FI of 3.0 and a median FQ of 0.037 (Table 2). Thus, an event reversal of 3 patients, or 3.7 % of the study population, would reverse significance for 50 % of outcomes related to volar locking plates vs. casting. In 21 out of 36 outcomes (58.3 %), the number of patients lost to follow up was greater than the fragility index.

Close

Fig. 2 PRISMA flow diagram showing identification, screening, and inclusion of eligible PubMed, Embase, and MEDLINE articles.

Export to PPT

Fifteen outcomes were statistically significant with a median FI of 2.0 and a median FQ of 0.029 (Table 2). Significant outcomes included five related to function/healing, six related to patient satisfaction, and four related to adverse events. There were twenty-one non-significant outcomes, resulting in a median FI of 4.0 and a median FQ of 0.042. These outcomes included ten related to function and healing, two related to patient satisfaction, and 9 related to adverse events.

The most common outcome category was healing/function, which included fifteen outcomes with a median FI of 3.0 and a median FQ of 0.067. Adverse event outcomes followed, totalling 17 instances and resulting in a median FI of 4.0 and a median FQ of 0.034. Finally, patient satisfaction outcomes totalled to 8, and had a median FI of 2.0 and a median FQ of 0.016 (Table 3).

Randomized trials demonstrated an decrease in fragility over time, with studies published before 2022 resulting in a median FQ of 0.029 (FI = 2.0), and studies published in 2022 or later resulting in a median FQ of 0.059 (FI = 3.0) (Table 4). Subanalyses were also conducted for trials where losses to follow up were greater than the FI. Among these studies, the median FI was 3.0, and the median FQ was 0.021. Studies that FIs greater than their losses to follow up resulted in a median FI of 3.0 and a median FQ of 0.057 (Table 5).

Cochrane bias assessment revealed that nearly all of the studies examined were found to be of “low risk”, even though the overall fragility of the studies was found to be high (Table 6). Those that were found to include potential bias demonstrated “some concern” due to deviations from intended interventions.

4 Discussion

The purpose of this study was to assess the statistical fragility of RCT outcomes examining the efficacy of VLP versus casting during the treatment of distal radius fractures. This is the first study, to the best of our knowledge, that analyzes these fragility metrics to quantify the statistical robustness in this field. Among the 36 outcomes assessed in this study, the median FI was 3 and the median FQ was 0.037, deeming these RCTs to be fragile on average. Thus, low statistical robustness may be a factor in the incongruity between level I evidence recommendations and surgeon practices concerning VLP and casting.

Although there is no gold-standard threshold of fragility metrics that signifies robustness, values reported in previous studies can serve as a relative guide. In a previous study examining the fragility of distal radius fracture treatment outcomes, Megafu et al. observed an FQ of 0.097, which was considered a fragile outcome.²⁸ Similar FQ values have also been deemed fragile in previous studies.^29,30 When analyzing fragility metrics, the FQ is particularly important because it can overcome limitations associated with the FI by accounting for sample size.³¹ Thus, this study will focus on FQ values for subsequent analyses.

There were 15 statistically significant outcomes included in this study, reporting a median FI of 2.0 and a median FQ was 0.029. Significant outcomes were more fragile than non-significant outcomes, which yielded a median FI of 4 and median FQ of 0.042 across 21 outcomes. This suggests the potential for type-1 error (alpha) in the included RCTs, which occurs when the null hypothesis is wrongfully rejected. Trafimow and Earp have previously explored the inability of the p-value to effectively account for type-1 error across scientific disciplines, meaning the fragility indices reported in this study may be more sensitive to robustness than p-values alone.³² The fragility of statistically significant outcomes in this study is also highly clinically relevant, as these outcomes often direct treatment decisions. For example, a RCT by Lawson and colleagues posited that volar plates were significantly more successful than casting at 3 months and 12 months post-operation, reporting p-values of 0.10 and 0.03, respectively.³³ However, FQs for these results tell a different story, as they resulted in only 1.3 % and 1.9 %, respectively. Ultimately, p-values reported alone may demonstrate misleading robustness, leaving readers with an exaggerated certainty of VLP superiority.

In a subgroup analysis of FQ, the most fragile outcome category was patient satisfaction (median FQ of 0.016), followed by adverse events (median FQ of 0.034) and healing/function (median FQ of 0.067). The marked fragility of patient satisfaction outcomes is not entirely surprising, as these outcome measures are often subjective and can be influenced by a variety of patient characteristics.³⁴ Murasko et al. highlighted the inconsistencies in patient satisfaction evaluation, calling to attention the lack of a correlation between patient satisfaction and surgical outcomes.³⁵ The fragility statistics calculated in this study corroborate the low statistical strength associated with patient satisfaction outcomes, and underscores the need for more standardized measures if they are to be included in RCT results. The notable fragility of outcomes pertaining to adverse events was also concerning, as potential for complications is a key aspect of intervention decision making. Although volar locking plates are often supported over casting for their low complication rates, these differences may be exaggerated.^36,37 Thus, VLP may be preferable to casting in more selective settings than previously thought, and potential complications of VLP may require more serious consideration from providers.^38,39

In over half of included outcomes, losses to follow up equaled or exceeded the FI or rFI, reiterating minimal statistical robustness especially when accounting for lost patients. Previous fragility studies have also reported high rates of this phenomenon, underscoring the need for study designs that minimize losses to follow up, especially when fragility indices are high.^40,41 In a secondary analysis of studies in which losses to follow up were less than reported fragility metrics, the median FI did not change, but the FQ increased from 0.021 to 0.057. Given this trend, it is likely that RCTs with large losses to follow up have particularly poor robustness, and should be identified by readers for this reason.

Finally, a temporal analysis of fragility revealed that papers published before 2022 were more fragile than studies published in 2022 or later. This trend is promising as it reflects an improved strength of RCT design over time. In 2010 the CONSORT statement for RCTs was revised, aiming to improve reporting of methods and results in RCTs.⁴² One intended effect of these changes was to allow trial design to be effectively replicated, improving the quality of scientific research. The temporal trends in this study may be explained by a continuation of efforts similar to CONSORT 2010, as increasing efforts to improve RCT strength likely result in decreasing outcome fragilities. As statistical robustness grew over time, so did the number of RCTs comparing VLP and casting. This trend may reflect an increasing interest in VLP treatment since its introduction in 2000.⁴³ With increased attention, trials in this field likely garnered improved resource access and were thus able to publish more robust findings. As the orthopedic community continues to evaluate its scientific rigor, this study proposes fragility analyses as an effective way of quantifying improvements or deteriorations in RCT quality over time.

5 Conclusion

The statistical fragility of outcomes examining VLP against casting for distal radius fractures proved fragile, with an overall median fragility quotient of 0.037, or 3.7 %. Statistically significant outcomes and patient satisfaction results were particularly fragile, and large losses to follow up were found to be an important contributor to RCT fragility. Although VLP is largely recommended by level I evidence, its superiority may not be as ubiquitous as argued by contemporary literature.

CRediT authorship contribution statement

Niklas H. Koehne: Conceptualization, Data curation, Formal analysis, Roles, Writing – original draft, Writing – review & editing, Project administration. Auston R. Locke: Data curation, Formal analysis, Project administration. Christoph A. Schroen: Data curation, Formal analysis, Roles, Writing – original draft, Writing – review & editing. Matthew Ramey: Roles, Writing – original draft, Writing – review & editing. Michael R. Hausman: Conceptualization, Supervision.

Guardian/patient consent

No patients were involved in this study. IRB approval was not required for this manuscript as only publicly available data was included in the investigation.

Ethical statement

N/A.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.