Section 2 of 7
Introduction
Hunter Bennett, Henry Blake, Noah d’Unienville, James Murray, and Jordan Fox · about 5 minutes
There is evidence suggesting researchers both knowingly and unknowingly adopt “questionable research practices” (QRPs) when designing, conducting, analyzing, and reporting research that can bias study results. Some of these practices include [1]: P-hacking, whereby researchers selectively analyze data or conduct multiple statistical tests until a significant result is found, often by adjusting variables or sample inclusion criteria; hypothesizing after the results are known (HARKing), whereby researchers formulate hypotheses on the basis of collected (and observed) data rather than testing prespecified hypotheses; and selective reporting, whereby researchers choose to only publish significant results while omitting null findings. The adoption of QRPs is likely to lead to an increase in the number of type I errors (i.e., false positives) reported in literature, contributing to publication bias and an inability to replicate prior research [2]. This has significant implications for the broader scientific community. Concerns about the replicability of previous research findings may erode public confidence and trust in science [3, 4]. Some have argued that this erosion of trust could fuel antiscience movements [5] and lead to a reduction in research funding [6]. An inability to replicate published research has been demonstrated in the field of psychology [7–10], cancer [11], economics [12], and philosophy [13].
In sports science-related topics, preliminary research has indicated that more than 80% of papers report hypotheses that are supported by study results [14]. This is noteworthy, as even if researchers did hypothesize with a “true” accuracy of 100% (which is very unlikely), for studies aiming to detect a significant effect, a minimum of 20% of those correctly hypothesized studies would be expected to produce nonsignificant results due to chance (false negatives; type II errors) when studies are powered at 80%, a commonly applied standard. This consideration is before factoring in many of the logistical constraints that are common in sports science-related research, such as low sample sizes [15], smaller than predicted effect sizes [16], and high measurement variability [17]—all of which would cause a further increase in the type II error rate. Similarly, a recent examination of almost 1700 articles in sports science-related journals reported a large excess of research with statistically significant results, with P-values of just below 0.05, a pattern not expected to occur if the published results were less biased [18]. Collectively, this may imply that sports science research is plagued with QRPs, highlighting the need for active prevention.
Preregistration has emerged as one possible method of mitigating QRPs [19]. Preregistration is the process of prospectively registering a research plan or study protocol prior to that study commencing. These outline the study’s hypotheses, methods, and analysis plans before data collection begins, increasing transparency and accountability in the research process. While there has been a widespread push for researchers to adopt open science practices [20], including preregistration [21], both the frequency and effectiveness of their uptake in sports science is unclear.
The adoption of preregistration varies considerably across research fields. For instance, recent non-peer-reviewed work reported that 43% of studies published in four quartile 1 (Q1) psychology journals had been preregistered [22]. Similarly, analyses of randomized controlled trials published in 2021 across 15 economics journals found a preregistration rate of 40% [23]. In contrast, an investigation of 156 epidemiological studies using data from the Norwegian Mother, Father, and Child Cohort (MoBa) found that only 1.3% had been preregistered [24]. To the authors’ knowledge, only one study has examined the prevalence of preregistered studies in sports science. That study, examining a small sample (n = 243) of articles published in Q1 journals, found that approximately 12% were preregistered [25]. However, it is also likely that preregistration differs between journal quartiles. Journal quartiles are rankings that divide journals within a specific subject category into four groups (Q1–Q4) on the basis of their impact or citation performance, whereby Q1 journals represent the top 25% of journals within that category, while Q4 journals represent the lowest 25%. For example, evidence from biomedical research suggests that randomized controlled trials (RCTs) are more likely to be published in higher-quartile journals [26]. Moreover, RCTs are also more likely to be preregistered [27], which suggests that the findings of the previous study may not accurately reflect preregistration practices across the broader field of sports science, including within lower-quartile journals, where RCTs are less common. Furthermore, the study did not investigate whether preregistration was associated with the rate of supported hypotheses, which would have given insight into their effectiveness in sports science.
Given the abovementioned research gaps, this study has three distinct aims: (1) to report on the proportion of original research studies that are preregistered in sports science journals, (2) to determine whether the proportion of supported hypotheses in preregistered studies will be significantly less than those that are not preregistered, and (3) to explore whether the frequency of preregistration and the proportion of supported hypotheses varies across different quartiles (i.e., Q1, Q2, Q3, and Q4, as identified via SCImago) of sports science journals. On the basis of the results of a recent systematic review reporting on the open science practices of Sport Medicine Research in a small (n = 243) sample of studies published in 2022, which indicated 12% of papers were preregistered [25], and assuming that the rate of preregistration has increased in sports science in line with other academic disciplines [23, 28], we hypothesize that 15–25% of included articles were preregistered. We also hypothesize that the proportion of supported hypotheses in preregistered studies will be significantly less than those that are not preregistered. Finally, we also hypothesize that higher-quartile journals will have a higher frequency of preregistered papers, and as such, a lower proportion of supported hypotheses, compared with lower-quartile journals.