Interview FUTURE-AI: Making AI in Healthcare Trustworthy!

Helmholtz Munich experts Prof. Julia Schnabel and Dr. Georgios Kaissis work in a team of 117 experts from 50 countries on this international project. In this interview, Julia Schnabel describes the necessity and goal of the project.

JS: The FUTURE-AI consortium unites 117 experts from 50 countries across all continents. Their aim is to define international guidelines for trustworthy healthcare AI in a structure and holistic manner. 

JS: There has been a significant amount of research in AI for healthcare over the past years, but only a limited number of AI tools have made the transition to clinical practice, due to major and persisting clinical, technical, socio-ethical and legal challenges, owing to limited trust and ethical concerns. These concerns relate to potential errors resulting in patient harm, biases and increased health inequalities, lack of transparency and accountability, as well as data privacy and security breaches.

JS: The international – and highly interdisciplinary - collaboration resulted in an international consensus guideline of many voices across different countries, effectively bringing in the different disciplines (data science, medical research, clinical medicine, computer engineering, medical ethics, social sciences) and healthcare data domains (radiology, genomics, mobile health, electronic health records, surgery, pathology). Through a modified and highly iterative Delphi survey approach conducted over a period of 24 months, the six guiding principles (FUTURE-AI) and a set of recommendations were developed and refined before reaching consensus (less than 5% disagreement between experts). We thus adhered to our own proposed guiding principles, notably fairness, transparency, and robustness, in bringing such a diverse set of international experts together in our consortium, and in seeking their consensus view.

JS: Not all six guiding principles may be needed for a specific medical AI tool. As an example, the Fairness principle is guiding a medical AI development team in identifying potential sources of bias already early in the development phase, to ensure the same performance of their medical AI tool across individuals or groups of individuals – including under-represented and disadvantaged groups. These biases may be due to individual patient attributes such as age, gender, ethnicity or due to differences in data origin like image scanners, hospital sites. It is particularly important that a medical AI tool works for the whole patient group that it is intended for and does not fail on some underrepresented subgroups. Fairness is in fact closely related to the Universality principle – the tool should be generalizable and not only work in a controlled training environment as well as the Robustness principle – the ability of a medical AI tool to maintain its performance and accuracy under expected or unexpected variations in the input data. 

Latest update: March 2024.