Rather than a rejection of the technology, the shift in enthusiasm among provider organizations from “transformational AI” to “incrementally valuable AI”
NEJM Catalyst
John League, CFA, Head of Digital Health Research, Advisory Board, Washington, District of Columbia, USA
March 16, 2022
cnas
Executive Summary
by Joaquim Cardoso MSc.
Chief Editor of “The Health Revolution” Institute blog / AI Health Unit
March 26, 2022
Provider organizations are far less optimistic about artificial intelligence (AI) than they were four years ago
… even though more of them than ever are using AI in clinical applications.
- This apparent tension raises important questions about the future of AI and machine learning (ML) in health care.
- Does flagging optimism in an environment of increasing AI adoption indicate a “tech-lash”-that is, a backlash against overhyped technology. Is it simply hard-earned and clear-eyed realism? Or is it a missed opportunity?
The Advisory Board Survey
- Advisory Board recently conducted a survey of strategic and analytics leaders at 250 provider organizations to learn about how those organizations are currently using and plan to use AI, similar to the 2018 survey edition
- Early insights from that data show that providers have less faith than in previous years that AI will power a transformation of health care. Many continue to struggle with data itself.
Provider organizations’ expectations for AI shift from transformational to incremental
Overall sentiment about AI has remained positive among provider organizations since 2018, although it has not grown.
- About 63% of respondents had positive expectations for AI in 2022,
- a similar proportion, 64%, in 2018.
However, the percentage of organizations expecting transformational change was cut nearly in half:
- only 19% expect AI to become transformational and essential at their organization,
- down from 37% in 2018.
Expectations for incremental value from AI climbed
- to 43% in 2022,
- from 27% in 2018 to
More provider organizations are also offering a negative assessment of AI than in 2018.
- In 2022, the response was 8%,
- an increase from 5% in 2018
Current use cases
- This shift occurs as more organizations are currently using AI in clinical and population health applications.
Provider organizations are already using AI for
- care variation reduction/protocol compliance (16%),
- risk and care gap identification (14%),
- precision medicine (18%), and
- creating new standards of care (14%).
More provider organizations than ever have plans to use AI across those domains in the future. The percentages of organizations with a committed plan to use AI for
- care variation reduction/protocol compliance (37%),
- risk and care gap identification (34%), and
- precision medicine (38%)
each exceeded their 2018 levels.
The long version of the articles provides insights about:
- Where are we going from here?
- Are we being too careful?
ORIGINAL PUBLICATION (long version)
Confronting the Reality of AI/ML in Care Delivery — and Advisory Board Survey
NEJM Catalyst
John League, CFA, Head of Digital Health Research, Advisory Board, Washington, District of Columbia, USA
March 16, 2022
Introduction
Provider organizations are far less optimistic about artificial intelligence (AI) than they were four years ago-even though more of them than ever are using AI in clinical applications.
This apparent tension raises important questions about the future of AI and machine learning (ML) in health care.
Does flagging optimism in an environment of increasing AI adoption indicate a “tech-lash”-that is, a backlash against overhyped technology?
Is it simply hard-earned and clear-eyed realism? Or is it a missed opportunity?
Advisory Board recently conducted a survey of strategic and analytics leaders at 250 provider organizations to learn about how those organizations are currently using and plan to use AI.
The survey asked many of same questions asked in a similar survey conducted in 2018, allowing us to compare how provider organizations’ perspective has evolved.
Early insights from that data show that providers have less faith than in previous years that AI will power a transformation of health care. Many continue to struggle with data itself.
At the same time, the least-adopted clinical or population health application of AI in 2022 is more widespread than the most-adopted application was in 2018.
Early insights from that data show that providers have less faith than in previous years that AI will power a transformation of health care. Many continue to struggle with data itself.
At the same time, the least-adopted clinical or population health application of AI in 2022 is more widespread than the most-adopted application was in 2018.
Table of Contents (TOC)
- Provider organizations’ expectations for AI shift from transformational to incremental
- Where are we going from here?
- Are we being too careful?
Provider organizations’ expectations for AI shift from transformational to incremental
Overall sentiment about AI has remained positive among provider organizations since 2018, although it has not grown.
- About 63% of respondents had positive expectations for AI in 2022, answering either “we believe AI will become a transformative, essential part of our health system” or “we believe AI will deliver incremental value.”
- A similar proportion, 64%, expressed a positive outlook on AI in 2018.
Overall sentiment about AI has remained positive among provider organizations since 2018, although it has not grown.
About 63% of respondents had positive expectations for AI in 2022, …A similar proportion, 64%, … in 2018.
However, the percentage of organizations expecting transformational change was cut nearly in half:
- only 19% expect AI to become transformational and essential at their organization,
- down from 37% in 2018.
However, the percentage of organizations expecting transformational change was cut nearly in half: only 19% expect AI to become in 2022 … down from 37% in 2018.
Expectations for incremental value from AI climbed
- from 27% in 2018 to
- 43% in 2022.
Expectations for incremental value from AI climbed from 27% in 2018 to 43% in 2022.
The least-adopted clinical or population health application of AI in 2022 is more widespread than the most-adopted application was in 2018.
The least-adopted clinical or population health application of AI in 2022 is more widespread than the most-adopted application was in 2018.
More provider organizations are also offering a negative assessment of AI than in 2018.
- Then, the share of organizations who did not expect to get value from AI was 5%. In 2022,
- that increased to 8%, and the increase was especially dramatic among small organizations (defined as those who actively care for fewer than 100,000 individuals annually).
In 2018, only 3% of small provider organizations said they did not expect to get value from AI.
That proportion has tripled, with 9% of small organizations saying in 2022 that they expect no value from AI.
More provider organizations are also offering a negative assessment of AI than in 2018.
Then, the share of organizations who did not expect to get value from AI was 5%.
In 2022, that increased to 8%,
This shift occurs as more organizations are currently using AI in clinical and population health applications.
Provider organizations are already using AI for
- care variation reduction/protocol compliance (16%),
- risk and care gap identification (14%),
- precision medicine (18%), and
- creating new standards of care (14%).
Four years ago, the most widely adopted use of AI in clinical care (care variation reduction/protocol compliance) was prevalent in only 12% of organizations.
That makes the least-adopted clinical or population health application of AI in 2022 more widespread than the most-adopted application was in 2018 (among the applications included in the survey).
This shift occurs as more organizations are currently using AI in clinical and population health applications.
More provider organizations than ever have plans to use AI across those domains in the future.
The percentages of organizations with a committed plan to use AI for
- care variation reduction/protocol compliance (37%),
- risk and care gap identification (34%), and
- precision medicine (38%)
each exceeded their 2018 levels.
The percentage with a plan to use AI for creating new standards of care slipped to 31%, but it had been the most-planned-for application in 2018 (33%).
More provider organizations than ever have plans to use AI across those domains in the future.
Where are we going from here?
Shifts in optimism are worrisome, but don’t approach the severity that would indicate a full-blown backlash against AI/ML.
Shifts in optimism are worrisome, but don’t approach the severity that would indicate a full-blown backlash against AI/ML.
Broader adoption and planning for use of AI suggest that provider organizations are not moving toward disinvestment-despite their generally thin margins and escalating labor costs.
That said, high-profile, overhyped proclamations about the transformational potential of AI have made commitment to AI harder, not easier.
The debacle of IBM Watson Health begs questions about the viability of AI in health care, even though the lesson to learn-especially now that Watson Health has been sold off in pieces-is that necessary ingredients needed for AI/ML have not developed at the same pace as the technology.
IBM gobbled up data and software applications, but that alone was insufficient to support the Watson AI.
Consider also numerous predictions of radiologists’ replacement with AI, which ignore the real and current value of the technology as an additive, augmentative tool for the radiologist.
… necessary ingredients needed for AI/ML have not developed at the same pace as the technology.
Rather than a rejection of the technology, the shift in enthusiasm among provider organizations from “transformational AI” to “incrementally valuable AI” most likely represents an increasingly nuanced understanding of the challenges around AI/ML-especially the challenges of making AI effective in the real world.
Rather than a rejection of the technology, the shift in enthusiasm among provider organizations from “transformational AI” to “incrementally valuable AI” …
As adoption increases, health care needs clear-eyed realism about two specific challenges of AI:
- working with data, and
- decision-making with data.
As adoption increases, health care needs clear-eyed realism about two specific challenges of AI: (1) working with data, and (2) decision-making with data.
The percentage of organizations expecting transformational change was cut nearly in half: only 19% expect AI to become transformational and essential at their organization.
Working with data
The lifeblood of AI/ML is data. The reality of health care data is that it is often most accurately described as “messy.”
The lifeblood of AI/ML is data. The reality of health care data is that it is often most accurately described as “messy.”
Overall, 16% of provider organizations say low-quality and/or poorly governed data is one of the most significant barriers to adopting AI or predictive modeling.
Overall, 16% of provider organizations say low-quality and/or poorly governed data is one of the most significant barriers to adopting AI or predictive modeling.
That’s especially true at large organizations (defined as those who actively care for more than 1.1 million individuals annually).
Although they tend to be more enthusiastic about AI (32% expect AI to be transformational and essential for their organization, compared to 19% of organizations overall), slightly more than 21% of them say that low-quality and/or poorly governed data is a significant barrier.
As such, one major challenge of working with data is that it just takes a long time to get good data.
As such, one major challenge of working with data is that it just takes a long time to get good data. Data cleaning and standardization from large data sets for use in AI training consistently requires a year or more of work.
Data cleaning and standardization from large data sets for use in AI training consistently requires a year or more of work. And that presumes that the necessary data is even accessible in the first place.
Overall, 10% of provider organizations say that limited access to needed data is a significant barrier.
Again, the larger the organization, the more acute that problem is: 15% of large organizations report limited of access to needed data as a significant challenge.
Overall, 10% of provider organizations say that limited access to needed data is a significant barrier.
Working with data also means recognizing the biases embedded in it.
Organizations should simply assume that any data used to train AI is biased. Structural bias is so strong in our institutions, our culture, and in the data that those create.
It is unrealistic to assume that our health care organizations generate unbiased data. It is much more likely that health care data is biased against those who are already underserved-low-income patients, rural patients, nonwhite patients, and non-cisgender patients.
Working with data also means recognizing the biases embedded in it.
It is much more likely that health care data is biased against those who are already underserved-low-income patients, rural patients, nonwhite patients, and non-cisgender patients.
Decision-making with data
Simply put, existing care delivery systems are not designed to incorporate input from AI/ML.
Simply put, existing care delivery systems are not designed to incorporate input from AI/ML.
Forget the science fiction of replacing a clinical decision maker like a physician with an algorithm; merely alerting the decision maker that the model has a suggestion is often an enormous task that requires significant changes to and remodeling of workflows.
In care delivery environments, those workflows are often complex, hierarchical, and dynamic in ways that the potential of an AI application does not anticipate.
Acknowledging the workflow implications of AI is a key first step to transformation, one without which any of the solutions to the challenges of working with data identified above will be for naught.
Are we being too careful?
Realistic expectations represent a genuine maturity in providers’ understanding and use of AI/ML.
The structural, systemic, and cultural changes required to take full advantage of AI/ML in health care are real and must be addressed proactively and incorporated into investment decisions.
That does not mean health care should abandon transformative aspirations.
AI/ML can have an outsized, positive impact on health care.
The challenges of total cost of care, administrative burden, clinician burnout, and patient experience have not responded to the non-AI solutions in health care’s toolkit.
Health care may worry about the cost of AI solutions, but the cost of not trying new tools like AI/ML may be even greater.
The challenges of total cost of care, administrative burden, clinician burnout, and patient experience have not responded to the non-AI solutions in health care’s toolkit.
Health care may worry about the cost of AI solutions, but the cost of not trying new tools like AI/ML may be even greater.
The prevailing challenge of AI/ML in health care is one of balance and trade-offs, like how to capture the potential of an application in the as-is reality of clinical workflows.
Health care organizations need to balance AI use cases that provide incremental value today with those that will be transformational in the future.
Health care organizations need to balance AI use cases that provide incremental value today with those that will be transformational in the future.
Originally published at https://catalyst.nejm.org on March 16, 2022.
