Causal language and inferences in observational rotator cuff database studies published from 2013 to 2022

Nadim Barakat; Brian C. Werner; Monica M. Arney; Wendy M. Novicoff; James A. Browne; J. Brett Goodloe

doi:10.1016/j.jor.2024.12.020

View/Download PDF

Buy Reprints

PDF

Translate this page into:

65 (); 106-111

doi:

10.1016/j.jor.2024.12.020

Causal language and inferences in observational rotator cuff database studies published from 2013 to 2022

Nadim Barakat^a,, Brian C. Werner^a, Monica M. Arney^a, Wendy M. Novicoff^b, James A. Browne^a, J. Brett Goodloe^c

a Department of Orthopaedic Surgery, University of Virginia, 2280 Ivy Road, Charlottesville, VA, 22903, USA

b Department of Public Health Sciences and Department of Orthopaedic Surgery, University of Virginia, 2280 Ivy Road, Charlottesville, VA, 22903, USA

c Department of Orthopaedic Surgery, Virginia Commonwealth University, 1250 E. Marshall Street, Richmond, VA, 23219, USA

⁎Corresponding author: Nadim Barakat. nb4tt@virginia.edu

Received: 2024-11-29, Accepted: 2024-12-17, Published: 2024-12-19

Professor P K Surendran Memorial Education Foundation

Disclaimer:
This article was originally published by Reed Elsevier India Pvt. Ltd. and was migrated to Scientific Scholar after the change of Publisher.

Abstract

The use of large multi-institutional databases in rotator cuff repair (RCR) research is expanding, but these studies are observational and cannot establish causation. This study examines the prevalence of causal language in clinical RCR database studies published from 2013 to 2022.

Administrative database and clinical registry studies on RCR published in eight orthopaedic journals from 2013 to 2022 were systematically identified and graded by two reviewers for the presence, absence, or inconsistent use of causal language in both the title/abstract and the full text. Chi-squared analyses were conducted to examine if there was an association between the gradings of articles and both the journal and year of publication.

Of 44 eligible articles, 14 were graded as consistently causal, 16 as inconsistent, and 14 as consistently non-causal. Chi-squared analyses revealed no statistically significant associations between the journal or year of publication and the title and abstract grading (p = 0.626, p = 0.277) or the full text grading (p = 0.374, p = 0.822).

Causal language was present in over two-thirds of observational RCR database studies published from 2013 to 2022. Authors should refrain from using causal language in database studies to prevent misleading readers and misinterpretation of findings.

Keywords

Causal language

Causality

Administrative databases

Database studies

Association

Rotator cuff repair

Show Related Articles from PubMed

1 Introduction

Arthroscopic rotator cuff repair (RCR) continues to increase at a significant rate. This may be the result of an aging population, increased number of outpatient procedures, or greater comfort in surgical intervention from both the surgeon's and patient's perspectives.¹^,² In addition to the rise of RCR rates, there has been a significant growth in RCR research, specifically originating from the United States and focused on arthroscopy.³^,⁴ One increasingly significant component of the literature on RCR is the use of large, multi-institutional administrative databases or clinical registries to answer clinical questions focused on patient risk factors and outcomes.⁵ Not only are database publications growing significantly in RCR research, they are becoming more common in all fields of orthopaedic surgery and growing at a faster rate than the overall number of orthopaedic research publications.⁶

Administrative databases are collections of data associated with patient care that track procedural complications, patient outcomes, and other variables primarily for billing purposes.⁷ Clinical registries are similarly comprised of data focused on patient care that has been collected for research purposes.⁸ These databases offer readily available and abundant data that can quickly be utilized to answer relevant clinical questions. In addition to being inexpensive, they can provide large sample sizes that cannot be easily obtained using other methods, potentially representative populations that are more generalizable than smaller randomized controlled trials, and the ability to track data over long periods of time.⁹ Despite being very useful for exploring a range of topics, database studies are observational, which restricts their conclusions.

For example, these retrospective studies cannot account for many relevant confounding variables, especially those not listed in the database utilized.¹⁰ These relevant confounders can include, but are not limited to, postoperative pain scores, socioeconomic status, occupation, or adherence to physical therapy. As a result, they cannot establish causal relationships between variables, only associations. This is important to recognize as cause-and-effect language in peer reviewed publications may be misleading to inexperienced readers who are unfamiliar with the limitations of utilizing these large databases. Thus, it is important to establish these database studies cannot establish causality.¹¹

Despite this serious limitation, many peer-reviewed database publications in the RCR literature describe their aims and findings in ways that suggest a cause-and-effect relationship between their variables of interest. The purpose of this study was to examine the use of causal language and inferences in clinical RCR studies using multi-institutional administrative databases or clinical registries published from 2013 through 2022 across eight orthopaedic journals. We sought to determine whether the use of causal language was associated with the journal and year in which the article was published in. We hypothesized that there would be a high prevalence of causal language and inferences in the RCR literature but that the use of causal language would not be significantly associated with certain orthopaedic journals or publication years.

2 Methods

2.1

2.1 Sampling and inclusion/exclusion criteria

The usage of causal language and inferences in database studies on RCR were analyzed in the following journals: The American Journal of Sports Medicine (AJSM), Arthroscopy: The Journal of Arthroscopic and Related Surgery (Arthroscopy), Journal of Bone and Joint Surgery (JBJS), Journal of Shoulder and Elbow Surgery (JSES), JSES International (JSES-I), JSES Open Access, Journal of the American Academy of Orthopaedic Surgeons (JAAOS), and Orthopaedic Journal of Sports Medicine (OJSM). These journals were determined to be inclusive of the most common journals to publish shoulder surgery database-type studies by expert opinion and prior research.⁵^,¹² The inclusion criteria were clinical retrospective studies that used multi-institutional administrative databases or clinical registries, articles that focused on RCR, and articles that were published in print during the calendar years of 2013–2022. Articles that used single institutional databases or those focused on shoulder arthroplasty, trends, finances, coding, or non-clinical topics were excluded. Articles published by reviewers of this study were also excluded to avoid any biases, and none were identified.

PubMed was searched for manuscripts published from each journal in 2013 through 2022 using the following search terms: ‘database OR registry’ and ‘rotator cuff.’ Of these publications, two reviewers manually reviewed each article to include those that followed the inclusion and exclusion criteria. See Fig. 1 for an example of the sampling process using Arthroscopy. This process was conducted for the other seven journals.

Article Sampling Process shows how articles published in Arthroscopy were obtained for analysis.

2.2

2.2 Assessment of database studies

The methodology for the grading criteria was adapted from and expanded upon by a prior publication in the British Medical Journal (BMJ).¹³ Two reviewers independently graded each title and abstract as one section and the full text as another section for causal language. Causal language was defined as when authors either made clear statements or inferences that implied that a certain exposure or variable would lead to a specific outcome or affect another variable. Also, making recommendations such as altering an exposure to result in a different outcome based on their observational findings was considered an example of a causal inference. Furthermore, the reviewers determined that providing an explanation to establish a causal relationship rather than suggesting a plausible relationship that requires additional controlled research to demonstrate causation as another example of a causal inference (See Table 1 for more examples). These definitions and examples are consistent with prior research focused on causation.^13–15

Table 1 Examples of clear causal language, causal inferences, and non-causal language.

Clear Causal Language	•Affect/Cause/Impact/Influence•Lead to/Result in/Responsible for•Effect/Role/Benefit•Increase/Decrease•Better/worse than
Causal Inferences	•Recommending Intervention•Controlling for confounders•Providing explanation to establish causality
Non-Causal Language	•Association/Correlation•Linked with/Predictive of•Describe Trends/Outcomes/Rates•Further research needed to investigate

The grading categories used were consistently non-causal (CN), inconsistent (IC), and consistently causal (CC). CN articles contained non-causal language without causal inferences throughout the entirety of the respective section. IC articles contained both causal and non-causal language. IC was defined as equal to or less than a 1:1 ratio of causal and non-causal language. CC articles were those that contained clear causal language or causal inferences throughout most of the section (>1:1 ratio) and communicated a causal conclusion or interpretation of the findings. Causal gradings were divided into CC and IC to distinguish between articles that rarely contained causal language compared to those that were consistent in their use of causal language. Gradings were context specific and not only focused on typical words that may convey causal relationships such as “effect.” The reviewers were attentive to what the language was referring to. Specifically, careful consideration was given to the title of the article, aims of the authors, and conclusions communicated to the readers (See Table 2 for examples and suggestions for non-causal language).

Table 2 Examples of statements and evaluations justifying section gradings and suggestions for improved language.

Grading	Claim 1	Claim 2	Evaluation	Suggestions
Consistently Causal	"Gout Can Increase the Risk of Receiving Rotator Cuff Tear Repair Surgery"	"Strict control of uric acid levels with hypouricemic medication may effectively reduce the risk of rotator cuff repair."	The authors use causal language when stating that a certain exposure could increase the risk of an outcome and infer a causal relationship when suggesting that manipulating the exposure could alter the risk of developing that outcome.	Gout is Associated with a Greater Risk of Receiving Rotator Cuff Tear Repair Surgery
Inconsistent	"The Effect of Sex Hormone Deficiency onthe Incidence of Rotator Cuff Repair"	"Sex hormone deficiency was significantly associated with RCR."	The authors are inconsistent when referring to their findings as associations versus causal relationships.	The Relationship between Sex Hormone Deficiency and the Incidence of Rotator Cuff Repair
Consistently Non-Causal	"Injections Prior to Rotator Cuff Repair Are Associated With Increased Rotator Cuff Revision Rates"	"This study strongly suggests a correlation between preoperative shoulder injections and revision RCR."	These authors conclude that their findings were that of an association or correlation rather than an effect or causal relationship.	No improvements are needed.

Each article's gradings were compared between the two reviewers. For articles where the two reviewers assigned different grades, a third expert reviewer (doctorate degree in research methods) was used as a tiebreaker in determining the final grade of an article. After the gradings were completed, claims from each article were retrieved and listed, and a summary of the evaluation was written to support the assigned grading (see Appendix for a comprehensive table).

This study did not involve any human participants and institutional review board approval was therefore not required. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

2.3

2.3 Statistical analysis

The percentages of articles given each grade, published in each journal and in each year, and utilizing either an administrative database or clinical registry were calculated. Chi-squared tests of independence were used to determine if there was an association between a section's grading and the journal and year in which it was published. In addition, a chi-squared test was used to examine the relationship between the grading of the title and abstract versus the full text grading. Lastly, the agreement between the primary two reviewers was assessed by calculating percent agreement and Cohen's kappa for both the title and abstract as one section and the full text as another section. Statistical significance was set to p < 0.05. Statistical analyses were performed using SPSS Statistics (version 28.0.1.1; IBM, Armonk, NY, USA). The reporting of this work adheres to the STROBE guidelines.¹⁶

3 Results

There were 44 eligible research papers. AJSM had four articles included (9.1 %), Arthroscopy had 20 articles (45.5 %), JBJS had two articles (4.5 %), JSES had six articles (13.6 %), JSES-I had three articles (6.8 %), JSES Open Access had one article (2.3 %), JAAOS had four articles (9.1 %), and OJSM had four articles (9.1 %). No articles were published in 2013 (0.0 %). One article was published in 2014 (2.3 %), one in 2015 (2.3 %), one in 2016 (2.3 %), five in 2017 (11.4 %), two in 2018 (4.5 %), 14 in 2019 (31.8 %), six in 2020 (13.6 %), seven in 2021 (15.9 %), and seven in 2022 (15.9 %). All 44 articles utilized administrative databases as their data source; no eligible articles used a clinical registry.

When evaluating the title and abstract of the included articles, 14/44 (31.8 %) were graded as CN, 16/44 (36.4 %) were graded as IC, and 14/44 (31.8 %) were graded as CC. When combining CC and IC, 30/44 (68.2 %) of articles had causal language present in the title and abstract (Table 3). When evaluating the full text of these articles, 14/44 (31.8 %) were graded as CN, 16/44 (36.4 %) were graded as IC, and 14/44 (31.8 %) were graded as CC. When combining CC and IC, 30/44 (68.2 %) of articles had causal language present in the full text (Table 4). A chi-square test of independence revealed there was a statistically significant association between the title and abstract final grading and the full text final grading, p < 0.001.

Table 3 Title and abstract grading of AJSM, Arthroscopy, JBJS, JSES, JSES-I, JSES Open Access, JAAOS, and OJSM.

Journal		Title and Abstract Grade			Total
Journal		CC	IC	CN	Total
AJSM		2	2	0	4
Arthroscopy		4	9	7	20
JBJS		1	1	0	2
JSES, JSES-I, and JSES Open Access		4	1	5	10
JAAOS		1	2	1	4
OJSM		2	1	1	4
Total	Count	14	16	14	44
Total	Percentage	31.8 %	36.4 %	31.8 %	100 %

Table 4 Full text grading of AJSM, Arthroscopy, JBJS, JSES, JSES-I, JSES Open Access, JAAOS, and OJSM.

Journal		Full Text Grade			Total
Journal		CC	IC	CN	Total
AJSM		3	0	1	4
Arthroscopy		4	9	7	20
JBJS		1	0	1	2
JSES, JSES-I, and JSES Open Access		4	3	3	10
JAAOS		0	3	1	4
OJSM		2	1	1	4
Total	Count	14	16	14	44
Total	Percentage	31.8 %	36.4 %	31.8 %	100 %

A chi-square test of independence revealed there was no statistically significant association between the journal and the title and abstract final grading (p = 0.626) nor was there a statistically significant association between the journal and full text final grading (p = 0.374). A chi-square test of independence revealed there was no statistically significant association between the publication year and the title and abstract final grading (p = 0.277), nor was there a statistically significant association between the publication year and full text final grading (p = 0.822).

For reviewer agreement, the two main reviewers had identical gradings for the title and abstract on 38/44 articles, or 86.4 % agreement. Cohen's kappa for the reviewers' gradings of the title and abstract was 0.771. The two main reviewers had identical gradings for the full text on 39/44 articles, or 88.6 % agreement. Cohen's kappa for the reviewers' gradings of the full text was 0.799. All the differences in gradings between reviewers were between CC and IC or CN and IC. There were never any different gradings between the two main reviewers in which a single reviewer graded a section as CC while the other graded it as CN.

4 Discussion

This study identified a large prevalence of causal language and inferences in clinical retrospective RCR studies using multi-institutional administrative databases in both the title and abstract (30/44) as well as the full text (30/44) of the articles. There were no studies identified that used a multi-institutional clinical registry, while common databases utilized included the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) and PearlDiver. The journal in which these articles were most frequently published in was Arthroscopy (20/44), and most articles were published from 2018 to 2022. There did not appear to be any relationship between the journal or publication year and the causality grading of an article. There was a strong association between the title and abstract and the full text's gradings, indicating consistency between language used across these two sections.

Retrospective studies using large multi-institutional databases are becoming more prevalent in all orthopaedic surgery and specifically within the RCR literature,⁵^,⁶ which is supported by this study demonstrating a significant surge within the most recent years compared to years prior. Consequently, it is becoming increasingly important for authors, readers, reviewers, and editors to be aware of the limitations and conclusions that observational studies can offer without controlling for relevant confounders. Observational research in general can only establish causal relationships in exceptional situations dependent on multiple stringent circumstances that these database studies do not satisfy.¹⁵^,¹⁷ Thus, they are only able to establish associations and should describe their aims and findings accordingly.

Establishing causality is still of value in medical research but can only be done in controlled conditions in which the effect of a variable on another can accurately be assessed, such as in randomized controlled trials (RCTs).¹⁸ However, RCTs have their own limitations such as the lack of generalizability to population groups not included, time and financial constraints, and feasibility or ethical concerns in which it may be inappropriate to study or manipulate a relationship such as in trauma.¹⁸^,¹⁹ Therefore, although database studies cannot establish causality, they remain incredibly important in providing information on relationships that could not otherwise be obtained.

Despite their value, authors conducting database studies should avoid the use of causal language and inferences and use clear non-causal language to avoid misinforming readers or leaving their results open to erroneous interpretation. In fact, there is evidence that causal language used for correlational findings can be a source of misinformation.¹⁵ Prior research in orthopaedics has shown that surgical residents have deficiencies in research interpretation, biostatistics, and identifying a study's design.²⁰ Meanwhile, similar findings have shown that college students and the general public have difficulty in interpreting if science reports can establish causality as well as differentiating between correlational and causal expressions.²¹^,²² When taking into account that the strength of causal language in research studies is often exaggerated when extrapolated into the media,²³^,²⁴ it is important for authors to be very clear and consistent in their language when reporting their findings to avoid having their conclusions being exaggerated and misinterpreted by others.

The use of causal language in observational research has been previously studied in the medical and orthopaedic literature. In the field of obesity and nutrition, one study found that 31 % of titles and abstracts of observational studies contained clear causal language.²⁵ In general orthopaedics, another study found that 60 % of titles and abstracts of observational studies published from 2016 to 2019 contained clear causal language.²⁶ However, it should be noted that these studies did not investigate a specific type of observational research and only included clear causal language, not causal inferences (See Table 1). Olarte Parra et al. reviewed 60 observational studies published in The British Medical Journal within a single year and found that causal language and inferences were present in 68 % of them.¹³ Thus, this study's results are consistent with prior research, but no studies published thus far have thoroughly investigated the RCR literature or surveyed studies published across such a significantly wide time span to identify differences in causal language over time.

4.1

4.1 Limitations

This study has several limitations. The sampling of articles was small, which limits the ability to identify any differences in the use of causal language across several journals or years of publication. However, this was primarily a limitation in data availability as database studies in RCR have started to increase mainly within the last few years.²⁷ Another limitation is that this study was limited to only clinical RCR articles published within the journals investigated and those using multi-institutional administrative databases or clinical registries, so the results may not be generalizable to all orthopaedic research or even all observational RCR research. Furthermore, many of the journals investigated are of American origin, so future work surveying orthopaedic journals based outside of the United States to see if this is an international trend could be beneficial.

Another limitation is the grading criteria in determining if a section was CN, IC, or CC. The thresholds for grading a section may have been slightly different among the two primary reviewers or even among different groups of reviewers not involved in this study. For instance, some authors would avoid causal language and inferences throughout the abstract and full text and even emphasize that their article could not establish causality in the limitations. However, multiple articles would use clear causal language in the title of the paper, resulting in a grade of IC. The graders tried to minimize this limitation by developing grading criteria before reviewing any of the articles, providing percent agreement and Cohen's kappa to quantify reviewer agreement for each section, and using an expert reviewer with a doctorate degree in research methodology to provide a final assessment when there was a disagreement between the primary reviewers. Furthermore, all disagreement between the primary reviewers involved classifying an article as IC or CC and IC or CN, never CC or CN, indicating consistent agreement on what was defined as causal language.

5 Conclusion

Causal language and inferences were present in more than two-thirds of observational RCR database studies published from 2013 to 2022. Authors should avoid the use of causal language in observational database studies to avoid misinforming readers and to reduce the risk of erroneous interpretation of their findings.

Patients consent

The current study did not require IRB approval nor Patient/Guardian consent because it did not involve human participants.

CRediT authorship contribution statement

Nadim Barakat: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing – original draft, Writing – review & editing. Brian C. Werner: Conceptualization, Formal analysis, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing. Monica M. Arney: Data curation, Investigation, Writing – original draft, Writing – review & editing. Wendy M. Novicoff: Conceptualization, Data curation, Investigation, Methodology, Supervision, Writing – original draft, Writing – review & editing. James A. Browne: Conceptualization, Methodology, Supervision, Writing – original draft, Writing – review & editing. J. Brett Goodloe: Conceptualization, Data curation, Supervision, Investigation, Methodology, Writing – original draft, Writing – review & editing.

Ethical statement

This manuscript has not been previously published and is not under consideration in the same or substantially similar form in any other peer-reviewed media. All authors listed have contributed sufficiently to the project to be included as authors, and all those who are qualified to be authors are listed in the author byline.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

References

Colvin A.C., Egorova N., Harrison A.K., Moskowitz A., Flatow E.L., . National trends in rotator cuff repair. J Bone Joint Surg Am. 2012 Feb 1;94(3):227-233.
[Google Scholar]
Zhang A.L., Montgomery S.R., Ngo S.S., Hame S.L., Wang J.C., Gamradt S.C., . Analysis of rotator cuff repair trends in a large private insurance population. Arthroscopy. 2013 Apr;29(4):623-629.
[Google Scholar]
Jin H., Ding Y., Wang W., et al . Arthroscopic therapy of rotator cuff diseases: a bibliometric study of the past 2 decades (2002-2021) Front Surg. 2022 Sep 15;9
[Google Scholar]
Lei L., Zhang C., Sun F.H., et al . Research trends on the rotator cuff tendon: a bibliometric analysis of the past 2 decades. Orthop J Sports Med. 2021 Jan 21;9(1)
[Google Scholar]
Weinreb J.H., Yoshida R., Cote M.P., O'Sullivan M.B., Mazzocca A.D., . A review of databases used in orthopaedic surgery research and an analysis of database use in arthroscopy: the journal of arthroscopic and related surgery. Arthroscopy. 2017 Jan;33(1):225-231.
[Google Scholar]
Karlson N.W., Nezwek T.A., Menendez M.E., Tybor D., Salzler M.J., . Increased utilization of American administrative databases and large-scale clinical registries in orthopaedic research, 1996 to 2016. J Am Acad Orthop Surg Glob Res Rev. 2018 Nov 16;2(11)
[Google Scholar]
Pugely A.J., Martin C.T., Harwood J., Ong K.L., Bozic K.J., Callaghan J.J., . Database and registry research in orthopaedic surgery: Part I: claims-based data. J Bone Joint Surg Am. 2015 Aug 5;97(15):1278-1287.
[Google Scholar]
Pugely A.J., Martin C.T., Harwood J., Ong K.L., Bozic K.J., Callaghan J.J., . Database and registry research in orthopaedic surgery: Part 2: clinical registry data. J Bone Joint Surg Am. 2015 Nov 4;97(21):1799-1808.
[Google Scholar]
Bohl D.D., Singh K., Grauer J.N., . Nationwide databases in orthopaedic surgery research. J Am Acad Orthop Surg. 2016 Oct;24(10):673-682.
[Google Scholar]
Kim S.C., Bateman B.T., . Methodological challenges in conducting large-scale real-world data analyses on opioid use in musculoskeletal disorders. J Bone Joint Surg Am. 2020 May 20;102(Suppl 1):10-14.
[Google Scholar]
Chiolero A., . Big data in epidemiology: too big to fail? Epidemiology. 2013 Nov;24(6):938-939.
[Google Scholar]
Goodloe J.B., Barakat N., Novicoff W.M., Browne J.A., Werner B.C., . The use of causal language and inferences in observational shoulder arthroplasty database studies. J Shoulder Elbow Surg. 2023 Sep;32(9):1918-1923.
[Google Scholar]
Olarte Parra C., Bertizzolo L., Schroter S., Dechartres A., Goetghebeur E., . Consistency of causal claims in observational studies: a review of papers published in a general medical journal. BMJ Open. 2021 May 20;11(5)
[Google Scholar]
Hernán M.A., . A definition of causal effect for epidemiological research. J Epidemiol Community Health. 2004 Apr;58(4):265-271.
[Google Scholar]
Thapa D.K., Visentin D.C., Hunt G.E., Watson R., Cleary M., . Being honest with causal language in writing for publication. J Adv Nurs. 2020 Jun;76(6):1285-1288.
[Google Scholar]
von Elm E., Altman D.G., Egger M., et al . Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ. 2007 Oct 20;335(7624):806-808.
[Google Scholar]
Hill A.B., . The environment and disease: association or causation? Proc Roy Soc Med. 1965 May;58(5):295-300.
[Google Scholar]
Simon S.D., . Is the randomized clinical trial the gold standard of research? J Androl. 2001 Nov-Dec;22(6):938-943.
[Google Scholar]
Beks R.B., Bhashyam A.R., Houwert R.M., et al . When observational studies are as helpful as randomized trials: examples from orthopedic trauma. J Trauma Acute Care Surg. 2019 Sep;87(3):730-732.
[Google Scholar]
Araoye I., He J.K., Gilchrist S., Stubbs T., McGwin G., Ponce B.A., . A national survey of orthopaedic residents identifies deficiencies in the understanding of medical Statistics. J Bone Joint Surg Am. 2020 Mar 4;102(5)
[Google Scholar]
Seifert C.M., Harrington M., Michal A.L., Shah P., . Causal theory error in college students' understanding of science studies. Cogn Res Princ Implic. 2022 Jan 12;7(1):4.
[Google Scholar]
Adams R.C., Sumner P., Vivian-Griffiths S., et al . How readers understand causal and correlational expressions used in news headlines. J Exp Psychol Appl. 2017 Mar;23(1):1-14.
[Google Scholar]
Haber N., Smith E.R., Moscoe E., et al . Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): a systematic review. PLoS One. 2018 May 30;13(5)
[Google Scholar]
Buhse S., Rahn A.C., Bock M., Mühlhauser I., . Causal interpretation of correlational studies - analysis of medical news on the website of the official journal for German physicians. PLoS One. 2018 May 3;13(5)
[Google Scholar]
Cofield S.S., Corona R.V., Allison D.B., . Use of causal language in observational studies of obesity and nutrition. Obes Facts. 2010 Dec;3(6):353-356.
[Google Scholar]
Varady N.H., Feroe A.G., Fontana M.A., Chen A.F., . Causal Language in observational orthopaedic research. J Bone Joint Surg Am. 2021 Oct 6;103(19):e76.
[Google Scholar]
Pugely A.J., Bozic K.J., . Editorial commentary: rising interest in "big data" in arthroscopy: is the juice worth the squeeze? Arthroscopy. 2017 Jan;33(1):232-233.
[Google Scholar]