NEJM Catalyst Innovations in Care Delivery 2022; 08
Stephani Amanda Lukasewicz Ferreira, RN, MS, Arateus Crysham Franco Meneses, Tiago Andres Vaz, MS, Otávio Luiz da Fontoura Carvalho, PharmaD, Camila Hubner Dalmora, MD, Daiane Pressoto Vanni, RN, Isabele Ribeiro Berti, MD, Fabiane Rocca, Márcio Pereira Ramos, and Rodrigo Pires dos Santos, MD, PhD
July 20, 2022
Executive Summary by:
Joaquim Cardoso MSc.
health transformation — research institute and strategy advisory
ai health unit
July 24, 2022
Summary
What is the context?
- Surveillance of health care-associated infection (HAI) is the foundation of infection control and one of the first steps in infection prevention.
What is the problem?
- Traditionally, however, surveillance is performed by infection control professionals (ICPs) who manually review patients’ records, searching for defined criteria. Such an approach leaves room for subjective interpretation, resulting in low interrater reliability.
- Moreover, depending on the surveillance method used — for instance, a search based on antimicrobial results — it may have low sensitivity.
What is the solution?
- In Brazil, leaders at Tacchini Hospital and Qualis, a startup that offers infection control advisory and antimicrobial stewardship, have developed a machine-learning-algorithm robot that has been demonstrated to be a reliable tool for identifying patients with HAIs using a semiautomated method.
What are the results?
- The performance of this infection surveillance assistant (ISA) robot shows optimal sensitivity, specificity, accuracy, and negative predictive values, and the precision (positive predictive value) is acceptable.
- The ISA robot identified more patients with HAIs than did the infection control manual surveillance reference.
- The time spent on patient review was also reduced compared with that spent on manual surveillance.
- The robot detected HAI in one of every two or three patients reviewed in the interface.
What are the next steps?
- The years of the Covid-19 pandemic have highlighted the problem of the shortage of health care professionals, including ICPs.
- Tacchini Hospital and Qualis aim to increase infection control efficiency, enabling these professionals to spend more time on inpatient wards, implementing care bundles, than handling office activities, such as manual surveillance.
What are the additional applications?
- In this study, the authors describe the implementation of semiautomated surveillance in a single center, but expanding the model for different patient scenarios and multiple centers should be the future for external validation of machine-learning surveillance.
- Such models have the potential for generalization because they do not depend only on fixed rules for HAI classification, but they can also learn from data sets in different patient population settings.
Key Takeaways:
- The project was driven by infection control professionals (ICPs), who are key to health care–associated infection mitigation. This enables the efficiency and effectiveness of the development and deployment of such an effort.
- The augmented intelligence provided by the infection surveillance assistant (ISA), an artificial intelligence robot, supports the ICPs, enabling them to dedicate more hours to higher-skill patient care delivery.
- A multidisciplinary team, including the hospital’s ICPs (local team members) and the Qualis team’s infectious disease physicians, nurses, pharmacists, and IT experts (central team members), should be involved in the effort. This facilitates sharing of the difficulties faced during the journey (research, implementation, and deployment phase), and the solutions and new ideas.
- An explanatory machine-learning model (random forest) and an easy-to-use interface are critical components for model explanation and health care workers’ acceptance.
- The implementation of the ISA robot has had no adverse impact on full-time-equivalent staffing for infection control, but it has enhanced care delivery by enabling ICP staff to be more present on the hospital wards performing on-site training and practices validation.
Where to Start (excerpt from the full version)
For organizations looking to undertake such an initiative, the authors offer several factors for consideration.
First, recognize that HCW acceptance is crucial for innovative initiatives.
Adoption of innovation is a challenge in most in health care settings where it can directly impact people’s lives.
In an HCW-driven project, the connections between parties and the trust constructed are the basis for the project’s success.
The development of the ISA robot was based on transparency, shared information and knowledge, and motivated teams who trusted each other.
During the implementation and deployment phases, the whole team presented and solved the problems.
Next, understand that acceptance will come in stages.
The project planning, itself laid out in phases, facilitated acceptance.
At first, the parties agreed to test and evaluate the ML algorithm. The specialists in infection control approved the ISA robot algorithm performance on the basis of epidemiology-accepted metrics. These results were peer reviewed and published.
An easy-to-use and self-explained interface for ICPs to use to evaluate patients is critically important in enabling the ICPs to see the value and accuracy of the model as they evaluate patients.
Finally, be sure to maintain a rigorous process for retesting and validating the ML algorithm and the interface in a real-world situation, together, during the implementation and deployment phase; …
… this will instill confidence and build acceptance by demonstrating performance improvement. In January 2022, we deployed the system for the hospital ICPs’ use.
This is an implementation of automated surveillance in a single center; expanding the model to different patient scenarios and multiple centers should be the future for external validation of ML semiautomated surveillance.
These ML models have the potential for generalization because they do not depend on fixed rules for classification and can learn from data set in different patient populations.
Selected images:
Originally published at https://catalyst.nejm.org on July 20, 2022.
About the authors & affiliations:
Stephani Amanda Lukasewicz Ferreira, RN, MS
Nurse Coordinator, Qualis, Porto Alegre, Brazil
Arateus Crysham Franco Meneses
IT Specialist, Qualis, Porto Alegre, Brazil
Tiago Andres Vaz, MS
Chief Technology Officer and Head of Artificial Intelligence, Qualis, Porto Alegre, Brazil
Otavio Luiz da Fontoura Carvalho, PharmaD
Pharmacy Coordinator, Qualis, Porto Alegre, Brazil
Camila Hubner Dalmora, MD
Partner, Medical Director, and Infectious Diseases Physician, Qualis, Porto Alegre, Brazil
Daiane Pressoto Vanni, RN
Nurse, Infection Control Committee, Tacchini Hospital, Bento Gonc¸alves, Brazil
Isabele Ribeiro Berti, MD
Infectious Diseases Physician, Tacchini Hospital, Bento Gonc¸alves, Brazil
Fabiane Rocca
IT Business Analyst, Tacchini Hospital, Bento Gonc¸alves, Brazil
Marcio Pereira Ramos
Hospital Administrator and Quality and Safety Manager, Tacchini Hospital, Bento Gonc¸alves,
Brazil
Rodrigo Pires dos Santos, MD, PhD
CEO and Founder, and Infectious Diseases Physician, Qualis, Porto Alegre, Brazil
About the organizations:
Tacchini Hospital is a general 251-bed hospital for clinical and surgical care in Brazil’s southernmost state of Rio Grande do Sul.
It serves a region of approximately 400,000 people; it has more than 11,000 admissions per year, and more than 16,000 surgical procedures are performed in the hospital each year.
The hospital has two ICUs, one for clinical patients and one for surgical patients, totaling 30 beds (26 beds for clinical and four beds for surgical patients).
The hospital attends medium- and high-complexity patients for all clinical specialties and executes surgical procedures for general surgery, pediatric surgery, gynecology, mastology and obstetric surgeries, oncology, neurology, traumatology, plastic, vascular, and urology surgeries.
The institution has its research center, called Instituto Tacchini de Pesquisa em Saude, to support the hospital and researchers in clinical, epidemiological, and innovative project studies.
Qualis, located in Porto Alegre, State of Rio Grande do Sul, is a startup in the field of infection control, antimicrobial stewardship, and patient safety that has been working to prevent infections and antimicrobial resistance through telemedicine in more than 30 hospitals in Brazil for more than 10 years.
