Reproducibility in modeling and simulation of the knee: Academic, industry, and regulatory perspectives

Stakeholders in the modeling and simulation (M&S) community organized a workshop at the 2019 Annual Meeting of the Orthopaedic Research Society (ORS) entitled “Reproducibility in Modeling and Simulation of the Knee: Academic, Industry, and Regulatory Perspectives.” The goal was to discuss efforts among these stakeholders to address irreproducibility in M&S focusing on the knee joint. An academic representative from a leading orthopedic hospital in the United States described a multi-institutional, open effort funded by the National Institutes of Health to assess model reproducibility in computational knee biomechanics. A regulatory representative from the United States Food and Drug Administration indicated the necessity of standards for reproducibility to increase utility of M&S in the regulatory setting. An industry representative from a major orthopedic implant company emphasized improving reproducibility by addressing indeterminacy in personalized modeling through sensitivity analyses, thereby enhancing preclinical evaluation of joint replacement technology. Thought leaders in the M&S community stressed the importance of data sharing to minimize duplication of efforts. A survey comprised 103 attendees revealed strong support for the workshop and for increasing emphasis on computational modeling at future ORS meetings. Nearly all survey respondents (97%) considered reproducibility to be an important issue. Almost half of respondents (45%) tried and failed to reproduce the work of others. Two-thirds of respondents (67%) declared that individual laboratories are most responsible for ensuring reproducible research whereas 44% thought that journals are most responsible. Thought leaders and survey respondents emphasized that computational models must be reproducible and credible to advance knee M&S.