The fact that an A.I.-powered solution is technologically feasible doesn’t mean that society will find its use morally and ethically acceptable
Fortune
BY FRANÇOIS CANDELON , RODOLPHE CHARME DI CARLO , AND STEVEN D. MILLS
February 4, 2022
GETTY IMAGES
Key messages
by Joaquim Cardoso MSc.
Chief Editor of the blog “The Digital Health Strategist”
Contact tracing apps have enjoyed, at best, mixed success worldwide, and have been a failure in the U.S.
- They haven’t helped much for one key reason: People don’t trust companies and governments to collect, store, and analyze personal data, especially about their health and movements.
- A.I. generates as much fear as it does hope today.
- No wonder several companies have opted to make their A.I.-related values and guidelines public, and to adhere to the principles of Responsible A.I. even though the latter aren’t crystal clear or universal yet.
We believe there’s a pressing need for social contracts between companies developing A.I. applications and stakeholders such as customers, investors, employees, and citizens.
- Given the different interests, values, and preferences among and between them, Responsible A.I. isn’t enough.
The concept of a social license — which was born when the mining industry, and other resource extractors, faced opposition to projects worldwide — differs from the other rules governing A.I.’s use.
- Social license can be defined as “the negotiation of equitable impacts and benefits in relation to its stakeholders over the near and longer term.
- It can range from the informal, such as an implicit contract, to the formal, like a community benefit agreement.”
- The social license isn’t a document like a government permit; it’s a form of acceptance that companies must gain through consistent and trustworthy behavior as well as stakeholder interactions.
- Thus, a social license for A.I. will be a socially constructed perception that a company has secured the right to use the technology for specific purposes in the markets in which it operates.
Companies cannot award themselves social licenses; they will have to win them by proving they can be trusted.
- As Morrison argued in 2014, akin to the capability to dig a mine, the fact that an A.I.-powered solution is technologically feasible doesn’t mean that society will find its use morally and ethically acceptable.
- And losing the social license will have dire consequences, as natural resource companies, such as Shell and BP, have learned in the past.
Three pillars of a ‘social license’: A social license, and the sanction that business will gain from one, will rest on three pillars:
(1) Fairness and transparency; (2) Benefit; and (3) Trust
Kicking off the conversation: To obtain a social license for A.I., business must enter into a frank and continuous dialogue with stakeholders. Companies can take five steps to kick off this critical conversation.:
(1) Identify stakeholders; (2) Communicate and educate; (3) Explain the A.I.’s inner workings; and (4) Manage Risks
Companies are bound to need a legal license from government; an economic license from investors; and, above all, a social license from stakeholders if they wish to use A.I. at scale.
FULL VERSION
In the spring of 2020, when the COVID-19 pandemic erupted, policymakers hailed contact-tracing apps as one of the most promising ways that digital technologies could help control its spread.
Governments could monitor people’s diagnoses and, later, their test results and locations partly through self-reporting; alert those infected as well as others with whom they had come in contact; and develop disease-spread scenarios to support decision-making by using apps and A.I.-powered platforms.
Apple and Google even announced a historic joint effort to develop technology that health authorities could use to build the apps “with user privacy and security central to the design.”
However, contact tracing apps have enjoyed, at best, mixed success worldwide, and have been a failure in the U.S.
They haven’t helped much for one key reason: People don’t trust companies and governments to collect, store, and analyze personal data, especially about their health and movements.
Although the world’s digital giants developed them responsibly, and the technology works as it is meant to, the apps didn’t catch on because society wasn’t convinced that the benefits of using them were greater than the costs, even in pandemic times.
They haven’t helped much for one key reason: People don’t trust companies and governments to collect, store, and analyze personal data, especially about their health and movements.
You don’t need a data-driven algorithm to conclude that A.I. generates as much fear as it does hope today.
Despite everyday applications such as Siri and Alexa, most people, individually and collectively, are still worried about how business will use the technology.
“Mark my words, A.I. is far more dangerous than nukes,” declared Tesla and SpaceX founder Elon Musk, four years ago.
No wonder several companies have opted to make their A.I.-related values and guidelines public, and to adhere to the principles of Responsible A.I. even though the latter aren’t crystal clear or universal yet.
Even so, as the COVID-19 contact-tracing platforms as well as several other A.I.-powered use cases — such as Microsoft’s Tay chatbot, Amazon’s hiring algorithms, Clearview A.I.’s vision recognition software, and A.I.2’s Ask Delphi — have demonstrated, adhering to Responsible A.I. is far from sufficient to ensure that people will endorse A.I.’s use.
That’s why we believe there’s a pressing need for social contracts between companies developing A.I. applications and stakeholders such as customers, investors, employees, and citizens.
Given the different interests, values, and preferences among and between them, Responsible A.I. isn’t enough. Companies have to earn a social license to use A.I. before they deploy the technology, and they have to ensure that they retain the social license over time.
Given the different interests, values, and preferences among and between them, Responsible A.I. isn’t enough.
The concept of a social license — which was born when the mining industry, and other resource extractors, faced opposition to projects worldwide — differs from the other rules governing A.I.’s use.
Academics such as Leeora Black and John Morrison, in the book The Social License: How to Keep Your Organization Legitimate, define the social license as “the negotiation of equitable impacts and benefits in relation to its stakeholders over the near and longer term.
It can range from the informal, such as an implicit contract, to the formal, like a community benefit agreement.”
The social license isn’t a document like a government permit; it’s a form of acceptance that companies must gain through consistent and trustworthy behavior as well as stakeholder interactions.
Thus, a social license for A.I. will be a socially constructed perception that a company has secured the right to use the technology for specific purposes in the markets in which it operates.
The social license isn’t a document like a government permit; it’s a form of acceptance that companies must gain through consistent and trustworthy behavior as well as stakeholder interactions.
Companies cannot award themselves social licenses; they will have to win them by proving they can be trusted.
As Morrison argued in 2014, akin to the capability to dig a mine, the fact that an A.I.-powered solution is technologically feasible doesn’t mean that society will find its use morally and ethically acceptable.
And losing the social license will have dire consequences, as natural resource companies, such as Shell and BP, have learned in the past.
Companies cannot award themselves social licenses; they will have to win them by proving they can be trusted.
Three pillars of a ‘social license’
A social license, and the sanction that business will gain from one, will rest on three pillars.
- Fairness and transparency
- Benefit
- Trust
Fairness and transparency. If business is to be answerable for its use of A.I., it must be able to justify the manner in which its algorithms work, and be able to explain the resulting outcomes to prove its A.I. algorithms are fair and transparent. An A.I.-based recruitment system, for instance, should be able to demonstrate that all the candidates that provided the same, or similar, responses to a question posed by the machine on different days received the same rating or score.
Benefit. Companies must ensure that stakeholders share their perception that the advantages of using A.I. are greater than the costs of doing so. They can measure the trade-offs at the individual, company, and societal levels, comparing the benefits of better health, convenience, and comfort in health care, for instance, with the potential downsides such as security, privacy, and safety. Society may not always sanction A.I.’s use, and business should be ready for that. For instance, at the starts of the pandemic, the idea of merging all the European Union members’ health care databases was mooted. It wasn’t obvious that the benefits would be greater than the costs of doing so, though, so the idea was quietly killed.
Trust. Society must be convinced that companies that plan to use A.I. can be trusted with data acquisition, and will be accountable for their algorithms’ decisions. Trust is critical for social acceptance, especially when the A.I. can work without human supervision. That’s one reason society has been slow to endorse self-driving automobiles. People believe that the industry incumbents can create reliable mechanical vehicles, but they’re not convinced of the former’s ability to develop digital self-driving technologies yet.
Kicking off the conversation
To obtain a social license for A.I., business must enter into a frank and continuous dialogue with stakeholders. Companies can take five steps to kick off this critical conversation.
- Identify stakeholders
- Communicate and educate
- Explain the A.I.’s inner workings
- Manage Risks
Identify stakeholders. The nature and the number of stakeholders that will determine the grant of a license for A.I. will depend on the system. Moreover, the stakeholders may have different objectives, which will complicate the process. When the German delivery company DHL deployed a new A.I.-based pallet-sorting system last year, it had to manage its employees, who would work with it, and shareholders, who financed it. By comparison, automobile manufacturers are seeking a social license for autonomous driving systems from a larger number of stakeholders that inclues automobile owners, drivers, regulators, and society.
Communicate and educate. Just as companies are quick to share the benefits of A.I. systems, they must make public all the potential risks. They should also communicate the processes and procedures that they have installed to mitigate and manage them. Google, for example, periodically updates Policies & Principles of A.I. at Google, a document that it keeps in the public domain. It serves as a living reference for anyone, including regulators, who wishes to challenge Google if they feel that it isn’t adhering to its own rules and values.
Explain the A.I.’s inner workings. Most companies still don’t build A.I. in such a way that they can always explain exactly how the algorithms work. CEOs should consider partnering with A.I. start-ups, digital giants, and think tanks to learn how to do so. Scientists such as those at DARPA’s Explainable A.I. program are already developing self-explainable A.I. — that is, machine learning techniques that provide a decision and a supporting explanation, but without sacrificing the former’s accuracy for the latter’s sake.
Manage Risks. As with any other business risk, companies should map, assess, and manage A.I.-related risks. Because of the technology’s ability to learn and become better by itself, companies must ensure that human oversight and technological guardrails are always in place. Otherwise, the consequences can be disastrous. The A.I.-based cybersecurity systems of a British online grocery vendor shut down its website in March 2020, for instance, confusing a sudden COVID-19-related surge in demand with a distributed denial-of-service attack. If the retailer‘s engineers had been monitoring the system, they could have helped salvage the reputation it lost that day.
Own the workforce implications. If companies wish to manage the social consequences of using A.I., they have no choice but to anticipate the employment-related issues that are bound to crop up. They must identify the surpluses and gaps in their workforces; upskill and reskill employees; and be transparent about the implications for their employees‘ compensation.
The quality of mercy, it is said, is twice blessed, but A.I. needs to be thrice blessed if business is to succeed in using the technology.
Companies are bound to need a legal license from government; an economic license from investors; and, above all, a social license from stakeholders if they wish to use A.I. at scale.
About the authors
François Candelon is a managing director and senior partner at Boston Consulting Group and the global director of the BCG Henderson Institute.
Rodolphe Charme di Carlo is a partner at Boston Consulting Group and an ambassador at the BCG Henderson Institute.
Steven D. Mills is a managing director, partner, and chief A.I. ethics officer at Boston Consulting Group.
Some of the companies mentioned in this column are current or past clients of BCG.
Names cited
Leeora Black and John Morrison, authors of the book The Social License: How to Keep Your Organization Legitimate
Originally published at https://fortune.com
