1 Introduction

In 2001, the Institute of Medicine Committee on Quality of Health Care in America published a set of strategies to transform the health care system into one that is patient-centered, efficient and equitable, and called for the widespread use of information technology to support shared-decision making in clinical settings.¹ Later, The Medicare Access and CHIP Reauthorization Act (MACRA) of 2015 was passed establishing reimbursement models based on quality measures including patient-reported outcome measures (PROMs),^2,3 which spearheaded the use of patient reported outcomes (PROs) both clinically and in research studies. Since then, the use of PROs in orthopaedic surgery has grown rapidly,^4,5 and there are now several orthopaedic PRO registries in use, including three exclusively for hip arthroscopy.^6–8

While the utility of PROs is clear, the routine capture of PRO data is subject to several sources of bias.⁹ Collection bias, for example, occurs when patients change their responses according to the mode of data collection, and has been demonstrated in hip arthroscopy.^10,11 Most notably, non-response bias occurs when patients fail to complete postoperative PRO surveys, and has been reported as high as 61 % 2 years after undergoing arthroscopy.¹² Regardless of the source, bias in sampling can impact the internal validity of the study design, ultimately limiting the conclusions drawn.¹³ Furthermore, when patterns emerge in the particular groups that are impacted by the sources of bias, external validity is also threatened. This has been noted within the orthopaedic surgery literature, where the proportion of African American and Hispanic patients reported is up to 3.5 times lower than census data.^14,15 Therefore, as orthopaedic care increasingly relies on PRO data, ensuring the quality and generalizability of that data is of utmost importance.

Few studies have assessed non-response specifically within hip arthroscopy. Patient-specific factors including younger age, male sex, black race, lower income, and lower socioeconomic status, as well as methodological factors including greater number of survey questions, have all been associated with decreased response rate following orthopaedic surgery.^16–18 A recent systematic review on response rates after orthopaedic surgery found that age, sex and race were the most frequently reported factors influencing response rate.¹⁹ However, only 3 of the 97 studies included specifically investigated hip arthroscopy. No studies measured response rates beyond 1 year postoperatively even though the majority of published outcomes data reports 2-year follow up. Therefore, a better understanding of survey response up to 2 years postoperatively in the rapidly growing field of hip arthroscopy is necessary to improve data collection, generalizability of data, and aid in interpretation of published findings.

The purpose of this study was to determine the preoperative factors associated with survey non-response 2 years after hip arthroscopy. We hypothesized that survey non-response would be associated with younger age, male sex, and black race. We also aimed to determine what, if any, methodological factors contributed to non-response at 2 years.

2 Methods

All patients undergoing arthroscopic hip surgery between October 2015 and March 2020 at a suburban surgery center were identified for possible enrollment into a prospective registry housed at a single academic institution.²⁰ Arthroscopic procedures performed included labral repair or debridement and/or osteoplasty for femoroacetabular impingement (FAI) syndrome. Patients were approached by study staff either in clinic or in the preoperative area to obtain voluntary informed consent prior to surgery. A standardized, equal attempt was made to make contact with each patient. Exclusion criteria included patients who did not have an active email address or required English translation services. Patients who were already active in the registry for a prior procedure were not approached a second time for enrollment. Ethical approval for the study was obtained from the local insititutional review board.

Patients enrolled before the day of surgery were sent an email containing a link to an electronic survey to complete within 7 days of their scheduled procedure. Patients enrolled on the day of surgery were given an opportunity to complete the electronic survey in a private area before the surgery. Surveys were completed independently by the patient, without any members of the research staff present, unless assistance was requested. All patients who had not completed the baseline survey by the time of surgery received three follow-up email reminders to complete the survey. Patients who failed to complete any items on the baseline survey within 7 days of surgery were removed from the study.

Patients were sent an automated email at 1 year and 2 years after surgery with a link to complete a follow-up survey corresponding to each timepoint. Patients who did not respond to the initial email were subsequently sent a series of 3 automated reminder emails. For the 2-year postoperative survey, patients were additionally sent 3 standardized text messages containing the survey link and given a minimum of 2 phone call reminders. Reminders were stopped if the patient completed the 2-year survey or elected to withdraw from the study. The 2-year survey remained open until 3 years after the surgery date, after which the survey was closed. At that time, patients who failed to complete any items on the survey or withdrew from the study were considered non-responders.

Survey items consisted of Patient-Reported Outcomes Measurement Information System (PROMIS) computer adaptive tests for six domains including Physical Function, Pain Interference, Fatigue, Social Satisfaction, Anxiety, and Depression.²¹ Pain was measured using a Numeric Pain Scale (NPS).²² Preoperative expectations were measured using the Musculoskeletal Outcomes Data Evaluation and Management System expectations domain (MODEMS).²³ Activity levels were measured using the Tegner Activity Scale and Marx Activity Rating Scale (MARS).^24,25 Survey items were delivered in sequence automatically with the option to pause and resume the survey at a later time, so long as the survey remained open. All sociodemographic information was self-reported. Surgical information, relevant surgical and medical history, comorbidities, body mass index (BMI), and concomitant medications were collected from the electronic health record. All study data were collected and managed using REDCap electronic data capture tools hosted at the home institution.²⁶ REDCap is a secure, web-based software platform that provides an intuitive interface for validated data capture for research studies.

Categorical variables were reported as frequencies and percentages, while continuous variables were reported as means and standard deviation. A Shapiro-Wilk test was used to assess the distribution of continuous variables. The majority of these variables did not fit a normal distribution; therefore, nonparametric tests were used. A Pearson Chi-Squared test, or Fisher's Exact Test when cell counts were less than 5, was used to compare categorical variables. A Wilcoxon Rank-Sum test was used to compare continuous variables. A two-tailed p-value of p < .05 was considered statistically significant. A backwards stepwise elimination logistic regression was used to identify predictors of non-response to the 2-year survey. Age, sex, and race were chosen as candidate variables a priori based on the prior literature. Additionally, response to the 1-year postoperative survey was chosen a posteriori based on results of the bivariate analysis. Race was treated as a binary variable in the model due to small sample sizes in each of the non-white categories. Variables were removed at each step based on contribution to the R², and the final model was limited by minimum Bayesian information criterion (BIC). The analysis was performed using JMP Pro, Version 17 software (JMP®, Version 17.0, SAS Institute Inc., Cary, NC).

3 Results

Overall, 145 arthroscopic hip procedures were screened for enrollment within the study timeframe. Of these, 99 patients (68 %) completed the baseline survey, were enrolled in the registry, and included in the study. There were 74 (75 %) responders and 25 (25 %) non-responders at 2 years postoperatively in the study cohort. The median number of attempts to contact a patient via a combination of email, text message, and phone call prior to response was 5 (IQR = 1–7). The median time to follow-up was 25 months (IQR = 24–27). The median time required to complete a survey was 12.1 min (range = 5–49 min). There was no difference in the amount of time required to complete the baseline versus the 2-year survey (Z = −130, p = .27).

Demographic characteristics of responders and non-responders are displayed in Table 1. Overall, 70 (71 %) patients were female, and there was a higher proportion of males in the non-responders group compared with the responders group (52 % vs 22 %, p = .01). The non-responders group also had a higher proportion of patients who identified as Asian, Black or African American, or multiracial (32 % vs 10 %, p = .01). There were no other statistically significant differences in demographics between the two groups, including age, ethnicity, or smoking status (p < .05).

Several methodological factors, including the attending surgeon, the date of and staff involved in the consenting process, time to complete the baseline survey, and response to the 1-year survey, were examined (Table 2). There was a higher proportion of patients who did not respond to the 1-year postoperative survey among non-responders to the 2-year survey (80 % vs 46 %, p = .005). There were otherwise no statistically significant differences between responders and non-responders at 2 years based on the methodological factors examined (p > .05).

Table 3 compares the mean baseline PROs between responders and non-responders. The mean baseline scores were similar between the groups, and there were no statistically significant differences in scores in any of the baseline PROMIS domains, NPS scores, activity scales, or MODEMS expectations (p > .05).

A logistic regression was performed to ascertain the effects of age, sex, race, and response to the 1-year postoperative survey on the likelihood of response to the 2-year survey (Table 4). Backwards stepwise elimination was used to reduce the model to two risk factors: race and response to the 1-year postoperative survey. The overall model was significant with an R² of 0.13. The results demonstrated that, holding response to the 1-year postoperative survey constant, patients who did not identify as white had 4.3 times the odds of being a non-responder to the 2-year survey than patients who did identify as white (OR = 4.3, 95 % CI [1.3, 14.4], p = .02). Additionally, holding self-reported race constant, patients who did not respond to the 1-year postoperative survey had 4.5 times the odds of being a non-responder to the 2-year survey than patients who did respond to the 1-year survey (OR = 4.5, 95 % CI [1.5, 13.8], p = .008).

4 Discussion

PROs are an important tool for shared-decision making, reimbursement, and research within orthopaedics. A comprehensive understanding of the factors contributing to non-response to PRO-based surveys in hip arthroscopy is important for minimizing attrition bias, maximizing data representation, and ensuring results are generalizable across all populations. The present study demonstrated that sex, race, and response status at 1 year postoperatively were associated with non-response at 2 years. In particular, when controlling for age and sex, patients who did not identify as white and who did not respond at 1 year were more likely to be non-responders at 2 years.

The finding that race is associated with survey response has been shown previously in several other settings including in general orthopaedic surgery,^17,27 knee surgery,¹⁸ knee and hip arthroplasty,^28,29 and knee arthroscopy.³⁰ Kunze et al. analyzed data from a hip arthroscopy registry and found race to be independently predictive of loss to follow-up at 2 years. They showed that patients identifying as African American had a 141 % increased likelihood of being lost-to follow-up, while patients identifying as other non-white races had a 42 % increased likelihood of being lost-to follow-up.³¹ Meanwhile, there is evidence that racial disparities have an effect on outcomes of orthopaedic surgery.^15,32,33 Lack of representation within PRO data, which forms the foundation for evidenced-based decisions, may further widen these disparities. This gap may be further widened by language barriers posed by surveys in written English, as in our study. Therefore, developing strategies to maximize follow-up in underrepresented groups in hip arthroscopy is important. Future studies are needed to better understand how to target subgroups in order to improve equity in representation and quality of data.

Of the methodological factors examined in this study, only response to the 1-year postoperative survey was associated with response at 2 years. Hutchings et al. studied several organizational factors related with survey response in a hip arthroplasty cohort and found no difference in response rates based on when the baseline questionnaire was administered or the recruitment rates of the recruiting provider.²⁸ Results from the Danish Hip Arthroscopy Registry found no difference in response rates at 1 year based on the geographical region where the service was provided.⁶ Kunze et al. found that failure to provide a phone number at baseline was predictive of loss to follow-up at 2 years.³¹ It is unclear whether failure to provide a phone number reflects a patient's general lack of interest in participating, or instead an impaired ability for study staff to contact the patient. Similarly, it is unclear whether response at 1 year reflects a patient's general willingness to respond, or indicates more stable contact information for obtaining follow-up. In the present study, two forms of contact (email and phone) were available for each patient. While it was not possible to differentiate between those patients who responded to email versus text message or phone call, all patients received four emails prior to attempting contact by text message or phone call. The median number of attempts to contact a patient prior to response was five, suggesting that approximately half of patients responded to email and half responded to text message or phone call. This underscores the importance of multiple forms of contact as one method for maximizing survey response postoperatively.

Previous studies have consistently found younger age^6,27,34 and male sex^27,30,31 to be associated with non-response. Non-responders in the present study were on average 5 years younger than the responders, however, this did not reach statistical significance. Additionally, while there was a higher proportion of males in the non-responders group, sex was not shown to be an independent predictor in the multivariable analysis. This is likely due to the smaller sample size in the present study which limits the ability to detect small differences on the basis of a single factor, and points to the multifactorial nature of non-response. While older patients report lower rates of technology use, younger patients are more likely to withhold medical information.³⁵ The requirement that patients had an active email address at the time of enrollment may therefore mask the effect of age on non-response in the present study. Additionally, the use of multiple methods of contact may have provided patients with the option to receive reminders and interact with study personnel in the manner that best suited to them, regardless of age or sex. Young patients have been shown to respond to text messaging at a higher rate than older patients.³⁶ However, this study population is a reasonably young group with a mean age of 34 years, so technology use differences may not apply to this cohort. Finally, there were no differences in baseline PROs between responders and non-responders. Results from the prior literature are mixed, with one study reporting differences in PROs with modest effect sizes³¹ and another reporting no differences.⁶ This is reassuring that the present literature on outcomes after hip arthroscopy has validity, but there remains concern on the risk of non-response bias for postoperative outcome analysis.

This study has several limitations. First, patients who were initially enrolled but failed to complete any items on the baseline survey were removed from the study and were not eligible for follow-up. This decreased the sample size of the study which limits some analysis. It is also possible that patients who do not respond initially are also less likely to respond at 2 years, and in such case the non-response rate would be underestimated. Second, patients who withdrew from the study prior to the 2-year follow-up were included as non-responders. This may not accurately represent what their true response status at 2 years would have been had they otherwise remained in the study. Third, patient contact information may have changed over the course of the 2-year study period, however, multiple forms of contact were available for each patient and contact information was verified via chart review at 2 years. Additionally, this is a younger urban population that may be more geographically mobile making follow up more challenging. Finally, this was a small sample at a single institution and the results may not be generalizable to other populations. Despite these limitations, this study identifies several important factors predictive of response to a survey-based institutional registry, and reproduces findings from other hip arthroscopy registries.

5 Conclusion

Patients who do not identify as white and who do not respond to follow-up at one year postoperatively are more likely to be non-responders at two years after hip arthroscopy, even when controlling for age and sex. Responders and non-responders at two years do not otherwise differ in their baseline PROs of physical function, pain, and psychosocial health. Although there are notable differences in those who do and do not respond to followup surveys, it is reassuring for the generalizability of results in hip arthroscopy research that baseline PROs are similar. Developing targeted strategies to improve data collection methods within subgroups of hip arthroscopy patients may enhance response rates and improve the generalizability of results.

CRediT authorship contribution statement

Dominic J. Ventimiglia: Conceptualization, Investigation, Methodology, Formal analysis, Writing – original draft, Writing – review & editing. Zachary Clark: Investigation, Data curation, Writing – original draft. Antoan Koshar: Investigation, Writing – review & editing. Michael A. McCurdy: Project administration, Investigation, Writing – review & editing. Alexandra Baker Lutz: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Writing – original draft, Writing – review & editing. Michael S. Rocca: Data curation, Writing – review & editing. R. Frank Henn: Investigation, Supervision, Writing – review & editing. Sean J. Meredith: Conceptualization, Investigation, Project administration, Supervision, Writing – review & editing.

Declaration of interest

None.

Level of evidence

Level III, retrospective cohort study.

Ethics approval

This study was approved by the Institutional Review Board (IRB) Committee at the University of Maryland, Baltimore (HP-00062261).

Consent

Informed consent was obtained from all patients for participation in the study.

Funding

This work was supported by a grant from The James Lawrence Kernan Hospital Endowment Fund, Incorporated (BL1941007WS).