Autonomous Clinical Monitoring Platforms Using Reinforcement Learning and Deep Neural Networks

Abstract

Clinical monitoring systems operate independently, but accuracy may suffer because intelligent input validation and alarm justification are not included. Following the architecture of a human-centric reinforcement learning framework, a clinical monitoring platform for hospital wards that uses deep neural networks as foundational components has been constructed. The platform can adapt to any clinical environment through real-time monitoring, context awareness of the physiological condition of patients, and online learning from actual mistakes. Hospital-acquired conditions remain major issues of quality in health-care delivery. These complications, at least in part, are related to the complexity of the clinical environment, which can be difficult to oversee. Autonomous systems can help close that supervisory gap, but existing clinical monitoring systems tend to operate independently. Although these systems are multivariable, their focus remains on detecting anomalies in the monitored signals and warning medical personnel in case something goes wrong. However, these alarms do not consider the operation of other clinical systems, which might justify an alarm without a real need—the so-called crying wolf. Reinforcement learning algorithms, at least in theory, are able to learn to achieve a goal by shaping their behaviour with the help of other intelligent agents operating in the same environment. To achieve this goal in the hospital ward environment, an architectural framework has been developed. The basic assumption is that patients in hospital wards should be in stable physiological conditions. The main task of the proposed autonomous clinical monitor is, therefore, to prevent deterioration of patients’ conditions. Although all monitored signals are relevant, the mission is not to detect every single anomaly but to provide intelligent supervision of all hospital clinical monitoring, telecommunications and warning systems. Most importantly, the proposed system is autonomous: it is able to monitor the context information of the environment and, on the basis of its physiological state, to shape the error feedback, both confirming and invalidating alarms raised by other systems. The entire process is conducted in real time. The developed platform is capable of complementing the current clinical monitoring systems autonomously, learning on the basis of its own mistakes, and its effectiveness has been verified in a closed-loop pipeline by combining it with state-of-the-art clinical monitoring, wireless telecommunication and alarm systems.