Researchers from the Universitat Politècnica de València (UPV), part of the ITACA Institute, have proposed a new generation of resilient health artificial intelligence that can adapt to uncontrolled changes and situations in the real world, and can even predict future situations more effectively and robustly. Their work has been published in the Journal of Medical Internet Research, one of the most significant journals in the digital medicine sector internationally.
«This new generation of resilient AI will be distinguished by its adaptability and reliability in the face of uncertainty, changes, or biases in information. These are common in the real-world contexts of AI use in healthcare, beyond the laboratory environments where it is designed, which poses significant challenges for the development of healthcare AI, its routine clinical use, and its regulatory frameworks,» highlights Carlos Sáez, researcher at the ITACA Institute and professor in the Department of Applied Physics at the Universitat Politècnica de València.
The new paradigm proposed by the BDSLab team at ITACA-UPV covers the four key phases of AI: training (when AI learns from data); prediction (when AI is applied to new cases it has not previously observed); production environment (the maintenance of AI once it is in routine use); and AI regulation (the laws and regulations that govern the safety and trust of AI).
«This new approach to AI can resiliently adapt to changes in the context of use, whether technical, such as changes in the information systems involved; or socio-health context, adapting quickly and effectively to unexpected situations like pandemics. Moreover, ensuring the trust of users in AI and the fundamental rights of people is a key aspect of this model,» highlights Carlos Sáez.
Key to the reliability of AI in clinical practice
According to Carlos Sáez, resilient AI would directly impact improving the trust and safety of health AI systems and would benefit data-based clinical decision-making for millions of people in Europe.
«The AI models would stay updated, avoiding obsolescence, and adapt quickly to changes, reducing costs and risks, the interactive support to decision-making with control of medical risks and fundamental rights supported by robust and ethical AI techniques, all optimise the link and trust between human and AI,» emphasises the researcher from BDSLab-ITACA at the UPV.
A practical example: prioritising emergency hospital calls
This new AI paradigm would, for example, help to prioritise incidents in a healthcare emergency centre more optimally and reliably by optimising the questions asked during calls in real-time.
«In cases of high uncertainty, the system would suggest the operator request the most decisive additional information from the caller to improve the reliability of the response. For example, a call is received about a 20-year-old woman presenting apparent respiratory difficulty, without any known chronic respiratory diseases reported; the system would autonomously request information about the use of medications, such as oral contraceptives, or recent anxiety attacks, and, if there are no new data, classify the case as high-risk to avoid possible biases,» explains Carlos Sáez.
Project KINEMAI
The work is part of the «Modelling the Kinematics of Information towards Change-Resilient Medical Artificial Intelligence (KINEMAI)» project funded by the State Research Agency – Knowledge Generation Projects 2022.
Furthermore, some of its results will be studied in the new course of the new Master’s in Biomedical Engineering called «Data Quality and Trustworthy Artificial Intelligence».
Reference
Sáez, C.; Ferri, P.; García-Gómez, J.M. Resilient Artificial Intelligence in Health: Synthesis and Research Agenda Toward Next-Generation Trustworthy Clinical Decision Support. Journal of Medical Internet Research 2024, 26. https://www.jmir.org/2024/1/e50295/
Source: UPV’s Communication Area