How to Make Deep Learning Algorithms in Computational Pathology More Reproducible and Reusable
The use of artificial intelligence (AI) in the field of pathology, especially in the early detection of diseases, has increased rapidly in the past decade. However, despite an ever-growing number of publications in the field, only few methods are reused by other researchers and even fewer have entered a clinical routine workflow. A team of Helmholtz Munich researchers now analyzed how to improve reusability and reproducibility of these deep learning algorithms and published their work as a Comment in Nature Medicine.
One of the most important limitations that prevent algorithms from being adopted more widely is low reproducibility and reusability. The lack of code accessibility makes it difficult even for machine learning specialists to reproduce results. Another limitation is the generalization gap: AI algorithms tend to perform worse on external data, e.g., from other data sources or labs, than on test data that is similar to the training data. Hence, it is critical for publications to include independent testing cohorts in their studies to prevent these pitfalls and to evaluate the benefit for clinical practice.
Improve sharing knowledge in the field of computational pathology
„Our goal is to increase the usability of algorithms developed by scientists”, first author Sophia J. Wagner says. Based on a literature review, the authors provide guidance to improve the reusability and reproducibility including recommendations for code publication and data usage. With this comment, the scientists want to make other researchers and scientific journals aware of the importance of making code and data accessible. „Sharing of data and code will help computational pathology advance more quickly and ensure that knowledge can be shared throughout the community more effectively“, Dr. Tingying Peng, one of the last authors explains. “Ultimately“, Dr. Carsten Marr, Director of the Institute of AI for Health at Helmholtz Munich and also last author, adds „the algorithms should optimize clinical workflows and promote patient health“.
Wagner, S.J., Matek, C., Shetab Boushehri, S. et al. Make deep learning algorithms in computational pathology more reproducible and reusable. Nature Medicine (2022). DOI: https://doi.org/10.1038/s41591-022-01905-0