Discussion

We found poor agreement in the willingness-to-randomize young patients with a meniscal tear to APM or meniscal repair. However, the majority of meniscal tears were graded as suitable for randomization by at least 30% of surgeons. Poor agreement was also observed with respect to which meniscal surgery should be performed, or if treatment should be surgical or non-surgical. The poor agreement suggests poor consensus on the preferred treatment of meniscal tears in young patients with isolated meniscal tear supporting the need for an RCT. It further suggests that the majority of isolated meniscal tears in young patients could be suitable for randomization.

Meniscal repair has gained popularity as it is believed to reduce the risk of knee OA in the long term, in contrast to APM which is associated with increased risk for knee OA [4,6,7,9,17]. However, all existing studies comparing meniscal repair and APM are observational studies with a high risk of confounding by indication, which can lead to a comparison of selected groups of patients with different prognoses. An RCT provides stronger support for which surgical procedure to prefer in young patients with isolated meniscal tears that are considered eligible for surgery.

Performing an RCT is challenging, due to orthopaedics surgeons’ preference for one or the other treatment option for each specific patient. Although most surgeons participating in our study considered the treatment choice (i.e. repair or APM) for a young patient with a meniscal tear to be clear, our study revealed that there was little consistency among which meniscal surgery should be performed. Importantly, there was also low agreement on which tear was suitable for randomization, suggesting that most isolated tears in young patients could potentially be randomized.

For most of the case videos, the surgeons did not agree whether they were willing-to-randomize the meniscal tear shown. Only for two case videos 24/27 raters agreed on the appropriateness to randomize. The videos contained two different types of meniscal tears (radial or longitudinal tears), and the patients underwent two different procedures (APM and meniscal repair, not known by the rating surgeons). Because there was only one radial tear, we cannot conclude that the agreement was based on this meniscal tear type. However, there were multiple longitudinal meniscal tears, and from our data we can confirm that orthopaedic surgeons did not agree on the willingness-to-randomize for all longitudinal meniscal tears.

Another interesting finding from these two case videos, #6 and #12, is that both tears were in a vascularized area, which was one of the main arguments for the surgeons not to be willing to randomize a meniscal tear. Nevertheless, most observers (24/27) were willing to randomize these meniscal tears.

For meniscal surgery, we found that for some of the case videos, orthopaedic surgeons were in favor of a certain type of meniscal surgery. For example, for case video #5, all surgeons selected meniscal repair. Despite this strong preference, 7 out of 26 surgeons were still willing to randomize to provide evidence on whether this was in fact the case. We see similar results in case videos #9, 12, and 18 for meniscal repair, and in case video #10 for APM. Although 24 out of 26 orthopaedic surgeons prefer APM in the case #10, 11 surgeons were willing-to-randomize this patient either to meniscal repair or APM. Thus, even if for some menisci there is large agreement on which type of meniscal surgery should be performed, there is a relatively poor agreement in the willingness-to-randomize. Also, in situations where 80% of the surgeons have a preferred surgery (either APM or meniscal repair), more than half of the surgeons is willing to randomize, implicating that there is still uncertainty about what is the optimal surgery. The choice for surgery type did not seem to depend on the years of experience as a knee surgeon.

We are aware that a wide range of meniscal tears have been included. The meniscal tears included reflect the criteria in the actual planned RCT and the real-world situation. Given the variability in responses across all included tear types, there is no evidence that our inclusion criteria explain the lack of agreement. Rather, it highlights that the decision whether a person is supposed to be randomized, should not be based on the surgeon's preferences, but rather strictly based on the inclusion criteria of the trial.

We also asked whether non-surgical treatment was preferred over surgery. It was a challenging question, as the rating surgeon was aware that the patient had already begun an arthroscopy. Nevertheless, there were still quite some raters who preferred a non-surgical treatment for tears that were operated on, although agreement was poor. It could be due to a general shift towards non-surgical management before opting for surgery, since a trial found no essential differences between the two treatment strategies [18].

Lastly, an interesting finding is that there is only minor overlap between the answer “no surgery” to the question of which meniscal surgery they would perform on that video case and “conservative” to the question whether they preferred non-surgical treatment over surgical treatment. It is thus unclear for which other treatment raters could have opted.

Our results suggest that clinical equipoise is present – meaning that there is little agreement on which surgery to prefer for which meniscal tear, which provides a strong argument for an RCT. Given the poor inter-rater agreement, the non-consensus can result in pseudo-randomization, if applied in a setting with common patient selection and inclusion/exclusion criteria, making causal inference feasible. Consequently, we suggest evaluating a “natural experiment” alongside any future RCT for orthopaedic surgeons who are not willing to randomize patients into one of the surgical approaches. The surgeons decide before the start of the study whether they are willing to randomize patients, or whether they are only comfortable by their own judgement. In this pilot study we observed poor consensus between surgeons in the choice for type of meniscal surgery, thus we consider the choice of treatment to resemble random assignment.

Only a few observational studies included patients between 18 and 40 years with an isolated meniscal tear, so the proposed experiment can provide valuable new insights [14,17]. The concept is innovative because it utilizes the natural variance in clinical judgment when applied in a well-defined clinical study to create a randomized effect in a non-randomized study setting, thereby enabling causal inferences that are not attainable in traditional observational studies where no natural experiment occurs. It also reflects real/world variability as it allows for inclusion of surgeons and patients who are not willing to participate in an RCT and is cost-effective. It's a creative adaptation to the challenges faced in surgical research. It could serve as a model for other areas where RCTs are challenging to implement due to strong practitioner preferences or other constraints.

Our study has some limitations. The case videos were short, and the surgeons were unaware of the clinical information of the patients while rating the videos. Only the in- and exclusion criteria of this study were known. Clinical information is an important part of the clinical decision-making process, and therefore we acknowledge that the information provided to surgeons to make a decision was not identical to usual clinical practice. However, we do believe for our primary study purpose, to determine whether an orthopaedic surgeon is willing to randomize a meniscal tear, the information was sufficient. One of the main arguments not to randomize a meniscal tear was the location of the tear (vascularized/non-vascularized area), which was clearly visible on the videos. There were 7 meniscal tears in the red-red zone, 8 in the red-white zone, and 4 in the white-white zone, making it plausible that vascularization of the meniscal tear was a factor influencing the willingness to randomize. Another limitation was the quality of the videos. Although the visibility was acceptable, it was difficult to obtain the full picture of the entire meniscus. One strength of our study is that we included 27 knee surgeons from different types of institutions (general hospitals, academic institutes, and private clinics), and showed them almost 20 different meniscus videos. Furthermore, our CI on our estimates were narrow with a clear interpretation of poor agreement.