Machine learning pioneer Andrew Ng argues that focusing on the quality of data fueling AI systems will help unlock its full power.
MIT Sloan Management Review
by Sara Brown
Jun 7, 2022
Structure of the publication
- Introduction
- Unlocking the power of AI
- How data-centric AI can help
- Conclusion
Edited by
Joaquim Cardoso Msc.
Health Management Transformation
Institute for better health, care, cost, and access — for all
June 20, 2022
The last 10 years have brought tremendous growth in artificial intelligence.
Consumer internet companies have gathered vast amounts of data, which has been used to train powerful machine learning programs.
Machine learning algorithms are widely available for many commercial applications, and some are open source.
Now it’s time to focus on the data that fuels these systems, …
… according to AI pioneer Andrew Ng, SM ’98, the founder of the Google Brain research lab, co-founder of Coursera, and former chief scientist at Baidu.
Ng advocates for “ data-centric AI,” which he describes as “the discipline of systematically engineering the data needed to build a successful AI system.”
Ng advocates for “ data-centric AI,” which he describes as “the discipline of systematically engineering the data needed to build a successful AI system.”
AI systems need both code and data, and “all that progress in algorithms means it’s actually time to spend more time on the data,” Ng said at the recent EmTech Digital conference hosted by MIT Technology Review.
Focusing on high-quality data that is consistently labeled would unlock the value of AI for sectors such as health care, government technology, and manufacturing, Ng said.
“If I go see a health care system or manufacturing organization, frankly, I don’t see widespread AI adoption anywhere.”
This is due in part to the ad hoc way data has been engineered, which often relies on the luck or skills of individual data scientists, said Ng, who is also the founder and CEO of Landing AI.
“If I go see a health care system or manufacturing organization, frankly, I don’t see widespread AI adoption anywhere.”
This is due in part to the ad hoc way data has been engineered, which often relies on the luck or skills of individual data scientists, …
Data-centric AI is a new idea that is still being discussed, …
… Ng said, including at a data-centric AI workshop he convened last December. But he pointed to some common problems he sees with data:
- Differences in labeling
- The emphasis on big data
- Ad hoc data curation.
i.Differences in labeling.
In fields like manufacturing and pharmaceutics, AI systems are trained to recognize product defects.
But reasonable, well-trained people can disagree about whether a pill is “chipped” or “scratched,” for example — and that ambiguity can create confusion for the AI system.
Similarly, each hospital codes electronic records in different ways. This is a problem when AI systems are best trained on consistent data.
…each hospital codes electronic records in different ways. This is a problem when AI systems are best trained on consistent data.
ii.The emphasis on big data.
A common belief holds that more data is always better.
But for some uses, especially manufacturing and health care, there isn’t that much data to collect, and smaller amounts of high-quality data might be sufficient, Ng said.
For example, there might not be many X-rays of a given medical condition if not that many patients have it, or a factory might have only made 50 defective cell phones.
But for some uses, especially manufacturing and health care, there isn’t that much data to collect, and smaller amounts of high-quality data might be sufficient, Ng said.
For industries that don’t have access to tons of data, “being able to get things to work with small data, with good data, rather than just a giant dataset, that would be key to making these algorithms work,” Ng said.
For industries that don’t have access to tons of data, “being able to get things to work with small data, with good data, rather than just a giant dataset, that would be key to making these algorithms work,”
ii.Ad hoc data curation.
Data is often messy and has errors.
For decades, individuals have been looking for problems and fixing them on their own.
“It’s often been the cleverness of an individual’s skill, or luck with an individual engineer, that determines whether it gets done well,” Ng said.
“Making this more systematic through principles and [the use of tools] will help a lot of teams build more AI systems.”
“Making this (data curation) more systematic through principles and [the use of tools] will help a lot of teams build more AI systems.”
Unlocking the power of AI
Some of these problems are inherent to differences between companies.
Organizations have different ways of coding, and factories make different products, so one AI system won’t be able to work for everyone, Ng said.
The recipe for AI adoption in consumer software internet companies doesn’t work for many other industries, Ng said, because of the smaller data sets and the amount of customization needed.
“I think what every hospital needs, what every health care system may need, is a custom AI system trained on their data,” Ng said.
“Same for manufacturing. In deep visual defect inspection, every factory makes something different. And so, every factory may need a custom AI model that’s trained on pictures.”
“I think what every hospital needs, what every health care system may need, is a custom AI system trained on their data,” Ng said.
But to date there’s been a focus on more multipurpose AI systems that unlock billions of dollars of value.
“I see lots of, let’s call them $1 million to $5 million projects, there are tens of thousands of them sitting around that no one is really able to execute successfully,” Ng said.
“Someone like me, I can’t hire 10,000 machine learning engineers to go build 10,000 custom machine learning systems.”
“I see lots of, let’s call them $1 million to $5 million projects, there are tens of thousands of them sitting around that no one is really able to execute successfully,”
“Someone like me, I can’t hire 10,000 machine learning engineers to go build 10,000 custom machine learning systems.”
Data-centric AI is a key part of the solution, Ng said, as it could provide people with the tools they need to engineer data and build a custom AI system that they need.
“That seems to me, the only recipe I’m aware of, that could unlock a lot of this value of AI in other industries,” he said.
Data-centric AI is a key part of the solution … as it could provide people with the tools they need to engineer data and build a custom AI system that they need.
“That seems to me, the only recipe I’m aware of, that could unlock a lot of this value of AI in other industries,” he said.
How data-centric AI can help
While these problems are still being explored, and data-centric AI is in the “ideas and principles” phase, …
… Ng said, the keys will likely be tools and education, including:
i.Tools to find inconsistencie
ii. Empowering domain experts.
i.Tools to find inconsistencies. Tools could focus on a subset — or “slice” — of data where there is a problem so programmers can make the data more consistent. Reasonable people might label differently, but this problem can be mitigated if areas of dispute are caught early and a common way of labeling is agreed on, Ng said.
ii.Empowering domain experts. In specialized fields, experts should be brought on board. For example, technologists training artificial intelligence to recognize different aspects of cells should ask cell biologists to label images with what they see — they know cells far better than the data engineers. “This actually enables a lot more domain experts to express their knowledge through the form of data,” Ng said.
Moving toward standardization is something to look at, Ng said, but physical infrastructure can be a limiting factor.
A seven-year-old X-ray machine will generate different entries than a brand new one, and there aren’t any practical paths to making sure every hospital uses machines from the same generation.
It’s also hard to standardize between a factory that makes car parts and one that makes candy.
“Heterogeneity in the physical environment, which is very difficult to change, leads to a very fundamental heterogeneity in the data,” he said.
“These different sorts of data need different custom AI systems.”
A seven-year-old X-ray machine will generate different entries than a brand new one, and there aren’t any practical paths to making sure every hospital uses machines from the same generation.
“Heterogeneity in the physical environment, which is very difficult to change, leads to a very fundamental heterogeneity in the data,” he said.
“These different sorts of data need different custom AI systems.”
Originally published at https://mitsloan.mit.edu on June 15, 2022.
