2.1 Study design

This is a retrospective observational cohort study. Reporting of the present study follows the STROBE Statement guidelines for reporting observational studies.²¹

The Ethics Committee of the Federal State of Hamburg, Hamburg, Germany was informed and approved this study (2021-10582-WF). The study was conducted according to the ethical standards in the 1964 Declaration of Helsinki and its later amendments. Patient informed consent was obtained in both clinics before PROM data collection took place.

2.2 Patients

For the present purpose the database of two German hospitals, Schoen Clinic Hamburg Eilbek and Schoen Clinic Neustadt, were searched. All patients underwent THR or TKR between January 2016 and December 2018 in one of the hospitals. The data of the patients were included for analyses if the patients submitted for primary TKR or THR; and if preoperative and postoperative PROM data were available. The patients were excluded from analysis if they were submitted for revision arthroplasty or in case PROM data was not available.

During the above-referred time period, 9482 patients underwent THR, of which 8182 submitted for primary THR. From the 4726 patients who filled in the WOMAC questionnaire at admission, only 2638 completed the WOMAC questionnaire at 12-months follow-up (FU), and the EQ-5D-5L at admission and 12-months FU. In the same time period 5747 patients underwent TKR, of which 5377 submitted for primary TKR. Of the 3158 patients who filled in the WOMAC questionnaire at admission, 1728 also completed the remainder of necessary PROM questionnaires.

Patients who submitted for THR were on average 68.4 years old, and majorly female (57%). 74% of the patients were classified into American Society of Anesthesiologists (ASA) group II. Only few patients (3.3%) experienced in-hospital complications. The mean length of hospital stay (LOS) was 7 days. Around 76% of the patients who underwent THR were mobilized during the first 6 h after surgery, in line with the hospitals’ enhanced rehabilitation program (ERP). Similar descriptive data was obtained for the patients in the TKR group. For further details see Table 1.

2.3 The WOMAC and the EQ-5D-5L questionnaires

The WOMAC evaluates three dimensions (pain, stiffness, and physical function) with the use of 24 items: 5 pain, 2 stiffness and 17 physical function items. It produces three subscale scores, one for each dimension, and a total index score.²² The Likert WOMAC version used in this work²³ rates on an ordinal scale of 0–4, with lower scores indicating lower levels of symptoms or physical disability. Each subscale is summed up to a maximum score of 20, 8, and 68 score points, for pain, stiffness and physical function, respectively. The WOMAC total index score is calculated by adding up the three sub-scores. The questionnaire is self-administered and takes 5–10 min to complete.

The EQ-5D-5L is a widely used generic measure of health-related quality of life (HRQoL). It assesses health in five dimensions (mobility, self-care, usual activities, pain/discomfort, anxiety/depression), each of which has five levels of response. Each health state can be assigned an index score based on societal preference weights,²⁴ ranging from less than 0 (where 0 is the value of a health state equivalent to dead) to 1 (the value of full health).

2.4 Data collection and procedures

PROM data collection took place in both clinics preoperatively at the admission day and at 3- and 12-months FU via postal survey. Despite PROM data collection being part of the routine procedure in both hospitals, it only takes place after patients gave their informed written consent. For organizational reasons, some patients could not participate in the PROM data collection. Additionally, some patients did not give their consent to PROM data collection. These two reasons can explain the number of patients that were admitted for surgery, but whose preoperative PROM data was not available for analysis. Missing data at 3- and 12-months follow-up was due to non-responders in the postal survey.

Basic data such as age and gender were taken from the patients' admission file. Length of stay (LOS), participation in the enhanced recovery program (ERP) and comorbidity index was taken from the hospitals' standard documentation. Additionally, data on the ASA classification score, surgery time and complications were retrieved from the annual compulsory German quality reporting, and PROM data was retrieved from hospitals’ standard PROM collection process. The data of both hospitals was exported and anonymized.

2.5 Definition of meaningful improvement (MI) and patient acceptable symptom state (PASS) thresholds

The meaningful improvement (MI) threshold assessed in this work is defined as the improvement necessary for a patient to benefit from surgical treatment. While MCIDs, first introduced in 1989 by Jaeschke et al.,²⁵ would reflect the minimum change in the WOMAC score that patients would perceive as improvement of their health status,²⁶ the MI is the WOMAC score improvement necessary to induce a change in Health Status Index that differentiates the patients who did not receive a benefit from surgery from the ones who did, regardless the amount of the improvement.

PASS²⁷ cut points were defined as the WOMAC score at certain points in time, for which patients perceived a minimal improvement of their health status.²⁸ MI and PASS values were calculated for all patients in the THR and TKR groups, and for stratified subgroups for 3- and 12-months FU times.

2.6 Statistical methods

Descriptive statistics were performed to characterize the sample. Multiple Ordinary Least Squares regression models were used to estimate determinants of WOMAC score improvements between admission and 12-months FU. The WOMAC score improvement was used as dependent variable. Several explanatory variables (preoperative WOMAC score, age, gender, in-hospital complications) were tested simultaneously.

Given the large impact of the preoperative WOMAC score on post-surgery WOMAC score improvement, subgroups were built based on preoperative WOMAC scores. Patients for both procedures (THR and TKR) were clustered in subgroups of 10-WOMAC admission score steps. Due to the need of sufficiently large subgroups, patients with the highest and lowest admission scores were clustered into larger subgroups (0–20; 71–96).

MI and PASS values were estimated with the use of the anchor-based receiver operator characteristics (ROC) curve method.²⁹ ROC curves are often used in medical research to evaluate clinical trials, or to identify thresholds for improvements in outcome measures, such as PROMs.^30,31 The plotted ROC curve can be used to determine MI or PASS thresholds that maximize sensitivity and specificity.³² When estimating MI and PASS values in orthopedic settings, the sensitivity and specificity are usually weighted equally.³³ In line with Froud et al.,²⁹ who compared available approaches, we have calculated thresholds by minimizing the sums of squares of 1-sensitivity and 1-specificity:threshold=min{(1−sensitivity)2+(1−specificity)2}

This approach picks the point on the ROC curve that is closest to the top left corner on the sensitivity – (1-specificity) plane. For the ROC curve to be plotted, an anchor differentiating between successful treatments and unsuccessful treatment must be set. For both, the calculation of MI and PASS, we set our anchor as an improvement in HRQoL, expressed through the EQ-5D-5L scores:ifEQ−5D−5L(12−monthFU)>EQ−5D−5L(admission)=1ifEQ−5D−5L(12−monthFU)≤EQ−5D−5L(admission)=0

Hence, we implied that for a patient to perceive a meaningful improvement (MI) in health status, a difference in HRQoL between baseline and 12-months FU must be detected. In line with King,³⁴ we tested the correlation between the variable used as anchor and the respective WOMAC score improvements and found that its correlation of 0.49 was larger than the required r = 0.3.

The statistical tests were carried out with the use of the software program Stata and its command rocmic.³⁵

3.1 Regression results

The strongest predictor of WOMAC score improvement at 12-months FU was the preoperative WOMAC total score for the patients in both groups (THR and TKR) (Table 2).

Based on the regression results, patients were stratified in subgroups according to their preoperative WOMAC total score (Table 3). The demographic characteristics of the patients in the THR and TKR groups were similar when both procedures were compared. With worse preoperative WOMAC total scores the share of female patients increased considerably, whereas the average age did not change regardless of subgroup association. Patients with worse preoperative scores had more comorbidities, endured more in-hospital complications, faced longer surgery times, stayed longer in the hospital, and were less early mobilized. For details see Table 3.

Patients in different preoperative WOMAC total score subgroups experienced different recovery paths (Fig. 1a and b). This affected the 3- and 12-months WOMAC total scores and their corresponding improvements. THR (TKR) patients in the subgroup “71–96” experienced an average improvement from admission to 3-month FU of 53.0 (47.4) score points. Their WOMAC total score improved by 52.6 (45.3) between admission and 12-months FU. In comparison, the patients in the subgroup “0–20” experienced only an average improvement of 8.0 (3.9) points from admission to 3-month FU and of 9.1 (6.3) points between admission and 12-months FU. Patients with higher (worse) WOMAC total scores at admission reported a much higher improvement in comparison to the patients with lower (better) preoperative WOMAC total scores (Fig. 1a and b).

Close

Fig. 1 Average and stratified (subgroup) WOMAC total score improvement in the (a) THR and (b) TKR groups.

Export to PPT

3.2 Meaningful improvement (MI) thresholds

The estimated overall and stratified MI thresholds by FU-time for the patients in the THR or TKR groups are presented in Table 4. The estimated overall MI for the patients in the THR group presented a WOMAC total score reduction (i.e. an improvement) by 27.0 at 3- and 28.1 at 12-months FU. Whereas the subgroups based on patients age and ASA classification did not show a considerable trend, the MI for the subgroups based on the preoperative WOMAC total score varied significantly. The estimated MI for THR patients with a low preoperative WOMAC total score (0–20) were −8.0 and −9.0 for 3- and 12-months FU, respectively. In contrast, THR patients with medium admission scores (41–50) had considerably larger MI values (−29.0 at 3- and −30.3 at 12-months FU).

Accuracy, expressed by the area under the curve (AUC), was systematically higher for the estimated MI values at 12-months FU. Whereas almost all estimated MI values for 12-months FU are considered acceptable (≥0.7) or excellent (≥0.8),³⁶ the majority of the estimated MI thresholds for 3-month FU lacked accuracy.

The estimated MI thresholds for patients in the TKR group were lower in comparison to the values of the THR patients, reflecting the general trend visible in Fig. 1b. TKR patients needed to improve by 18.7 and 17.8 WOMAC total scores points to experience a MI in their HRQoL. The estimated MI thresholds for the subgroups showed a similar pattern as for the THR patients. Moreover, when the MI thresholds were estimated based on the ASA classification subgroups, the results yielded an inconclusive trend. The estimated MI thresholds of the patients in the “ASA I & II” group were slightly larger than for patients with “ASA II and higher”. Interestingly, the MI thresholds for subgroups clustered by LOS showed that at 12-months, TKR patients who stayed longer than seven days demanded a lower improvement in WOMAC scores (MI: 16.0 to −18.0) than patients who were discharged earlier. The achieved accuracy of MI thresholds for the patients in the TKR group was lower in comparison to the values for the patients in THR group. Also, the achieved accuracy for 3-months FU MI was lower (AUC <0.7) than for 12-months FU, which for most cases was either acceptable or excellent.

Fig. 2a and b shows the share of patients in the THR and TKR groups that achieved their unstratified (overall) and their subgroup-specific MI thresholds at 12-months FU. While the unstratified 12-months MI of 28.1 for THR patients (17.8 for TKR) would only be achieved by 0–6% (6%) of the patients included in the subgroup “0–30” (”0–20”), it would be achieved by 80–85% (80–86%) of the patients in the subgroup “61–96”. Alerts for critical recovery paths based on unstratified MI would be significantly more sensitive for patients with lower admission WOMAC total scores. On the contrary, MI thresholds estimated based on preoperative WOMAC score subgroups would lead to an even distribution of patients achieving their threshold.

Close

Fig. 2 Overall and stratified meaningful improvement (MI) thresholds for WOMAC total score improvements at 12-months FU for (a) the THR and (b) TKR groups.

Export to PPT

3.3 Patient acceptable symptom states (PASSs) thresholds

The estimated overall and stratified PASS thresholds are presented in Table 5. The unstratified PASS threshold of 13.3 for THR patients (15.4 for TKR) at 12-month FU would result in an unfair distribution of patients achieving this target (see Appendix material Figures A1 and A2). PASSs thresholds at 3- and 12-months FU were dependent on pre-operative WOMAC scores. For THR patients in the “71–96” subgroup it was already acceptable to reach a WOMAC score of 28.0 at 12-months FU, whereas THR patients in the “0–20” subgroup required a PASS of 5.0. Analogue, TKR patients in the “71–96” subgroup accepted a WOMAC score of 45.0 at 12-months FU, whereas TKR patients in the “0–20” subgroup required a PASS of 6.0 (Table 5). Similar to MI thresholds, the accuracy of estimated PASS values was higher for THR patients and at 12-months FU. The unstratified PASS values at 12-months FU were less accurate (AUCs <0.7) than the admission score group-specific PASSs (most AUCs ≥0.7).

4.1 Study limitations

Our sample consists of patients that have responded to the WOMAC and EQ-5D-5L questionnaires at admission and 12-month post-surgery. Around 60% of THR and TKR patients treated in both hospitals were not included in the analysis because of missing PROM data in at least one of the data points. The reasons for missing data were elaborated on previously. Analyses showed that our sample of responders differed slightly from the full sample. The patients in the sample had slightly better health status at admission and less comorbidities. This source of bias, which was induced by missing PROM questionnaires, might have led to an overestimation of MI and PASS thresholds. Furthermore, our analyses were based on data of only two hospitals, a specialized endoprosthetic center and a general hospital. Moreover, we have observed that the accuracy of MI and PASS thresholds increased with the number of observations. Sample size should become more important when future studies attempt to integrate even more patient-specific factors, dividing the sample in granular subgroups.

In line with the results by Berliner et al.³⁷ our analysis showed that admission scores can significantly predict the WOMAC score improvements, and therefore, they can be used to cluster the patients into subgroups with homogenous recovery paths. Our results showed that THR and TKR patients, who were admitted to the hospital with a better joint-related health status, experienced less improvement in WOMAC total scores but a better symptom state at 3- and 12-months FU in comparison with patients with worse preoperative joint-related health status. Not surprisingly, a generic one-size-fits-it-all approach to thresholds leads to an uneven distribution of patients achieving their corresponding MI or PASS values. Both, for patients in the THR and TKR groups, cases with high (worse) preoperative WOMAC total scores were unlikely to reach the unstratified (overall) PASS but almost all patients with higher WOMAC admission scores reached the unstratified (overall) MI thresholds. At the same time, patients with low WOMAC admission scores have a high chance to reach the unstratified PASS, but not the unstratified MI threshold. When MI and PASS thresholds for subgroups were calculated (clustered by admission WOMAC total score) the estimated thresholds were better tailored to individuals. Our results showed that THR and TKR patients with higher (worse) preoperative WOMAC total scores need more WOMAC score improvement to achieve a meaningful change in their HRQoL and consequently benefit from surgery, compared to the patients with lower WOMAC admission scores. The opposite was confirmed for PASS values. Moreover, clustering patients based on ASA classification, patients age or LOS only yielded marginally differentiated MI thresholds. The accuracy of MI and PASS thresholds was highly dependent on the sample size and samples’ homogeneity considering the recovery paths.

Various studies have investigated thresholds for joint-specific PROMs, such as the WOMAC score.⁸ While one study calculated multiple kinds of thresholds,²⁶ the majority of studies concentrated on calculating MCID or PASS values, either by using an anchor-based and/or a distribution-based method. Dependent on the method used, a large variation in the MCID and PASS values could be observed. Terwee et al.³⁸ reported MCID values for the WOMAC pain sub-score at 6-months after THR (TKR) of 8.3 (13.3) when using the distribution-based method and of 22.4 (29.4) when using anchor-based ROC curves. Moreover, McKay et al.⁸ described a large heterogeneity within thresholds using the anchor-based method, which depended on (1) the country in which the investigation took place,^38,39 (2) the type of anchor used⁴⁰ and (3) the approach used to calculate the threshold.²⁹ Future research requires the determination of a “gold standard” method for calculating MCID/MI and PASS thresholds, without disregarding a differentiation of the threshold's goal (e.g., measuring treatment success vs. minimal improvement in health status). In the present study we have calculated MI to detect treatment success by estimating thresholds that differentiate patients that benefited from surgery from the ones who did not. Regardless, we observed that unstratified MI and PASS thresholds for the THR and TKR groups were in line with those reported in systematic reviews.^8,41

However, most studies estimated unstratified MCID and PASS thresholds and did not consider that the amount of improvement necessary for a patient to benefit from surgery might vary with patient characteristics. Davis et al.⁴² showed that failing to take into account admission scores when calculating thresholds can result in patients being misclassified as not having benefitted from surgery. Consequently, Berliner et al.^18,37 reported that the share of patients achieving an unstratified MCID depends on their admission score. Despite several studies emphasizing this effect, there is only limited literature considering it when calculating MI thresholds.⁴³