the health strategist
institute for continuous transformation
in health and tech
Joaquim Cardoso MSc
Founder and Chief Researcher & Editor
March 17, 20234
EXECUTIVE SUMMARY
OpenAI recently announced the release of GPT-4, an AI-powered language model with improved capabilities, including the ability to generate images and computer code from a prompt.
- While researchers are excited about the model’s potential to transform science, some scientists are frustrated that they cannot access the technology, its underlying code, or information on how it was trained, making it less useful for research and raising concerns about its safety.
- OpenAI has addressed concerns by disincentivizing GPT-4 to create dangerous, illegal, or damaging content, and has improved safety in the latest version, but some scientists argue that it is impossible to improve safety without access to the data used for training.
Scientists are calling for urgent guidelines to govern how AI and tools such as GPT-4 are used and developed, and they are convening an invitational summit at the University of Amsterdam on 11 April to discuss these concerns.
DEEP DIVE
GPT-4 is here: what scientists think
Researchers are excited about the AI — but many are frustrated that its underlying engineering is cloaked in secrecy.
Nature
Katharine Sanderson
March 16, 2023
Artificial intelligence company OpenAI this week unveiled GPT-4, the latest incarnation of the large language model that powers its popular chat bot ChatGPT. The company says GPT-4 contains big improvements — it has already stunned people with its ability to create human-like text and generate images and computer code from almost any a prompt — and researchers say that its abilities have the potential to transform science. But some scientists are frustrated that they cannot yet access the technology, its underlying code or information on how it was trained. That raises concern about the technology’s safety and makes it less useful for research, say scientists.
GPT-4, released on 14 March, can now handle images as well as text. As a demonstration of its prowess, Open AI, which is based in San Francisco, California, says that it passed the US bar legal exam with results in the ninetieth centile, compared with the tenth centile for the previous version of ChatGPT. But the tech is not yet widely accessible — only to paid subscribers to ChatGPT.
“There’s a waiting list at the moment so you cannot use it right now,” Says Evi-Anne van Dis, a psychologist at the University of Amsterdam. But she has seen demos of GPT-4. “We watched some videos in which they demonstrated capacities and it’s mind blowing,” she says. One instance, she recounts, was a hand-drawn doodle of a website, which GPT-4 used to produce the computer code needed to build that website, as a demonstration of the ability to handle images as inputs.
But there is frustration in the science community over OpenAI’s secrecy around how and what data the model was trained, and how it actually works.
“All of these closed-source models, they are essentially dead-ends in science,” says Sasha Luccioni, a research scientist specializing in climate at HuggingFace, an open-source-AI community. “They [OpenAI] can keep building upon their research, but for the community at large, it’s a dead end.”
Andrew White, a chemical engineer at University of Rochester, has had privileged access to GPT-4 as a ‘red-teamer’: a person paid by OpenAI to test the platform to try and make it do something bad. He has had access to GPT-4 for the past six months, he says. “Early on in the process, it didn’t seem that different,” compared with previous iterations.
He put to the bot queries about what chemical reactions steps were needed to make a compound, predict the reaction yield, and choose a catalyst. “At first, I was actually not that impressed,” White says. “It was really surprising because it would look so realistic, but it would hallucinate an atom here. It would skip a step there,” he adds. But when as part of his red-team work he gave GPT-4 access to scientific papers, things changed dramatically. “It made us realize that these models maybe aren’t so great just alone. But when you start connecting them to the Internet to tools like a retrosynthesis planner, or a calculator, all of a sudden, new kinds of abilities emerge.”
And with those abilities come concerns. For instance, could GPT-4 allow dangerous chemicals to be made? With input from people such as White, OpenAI engineers fed back into their model to disincentivize GPT-4 to creating dangerous, illegal or damaging content, White says.
Outputting false information is another problem.
Luccioni says that models like GPT-4, which exist to predict the next word in a sentence, can’t be cured of coming up with fake facts — known as hallucinating. “You can’t rely on these kinds of models because there’s so much hallucination,” she says. And this remains a concern in the latest version, she says, although OpenAI says that it has improved safety in GPT-4.
Without access to the data used for training, OpenAI’s assurances about safety fall short for Luccioni. “You don’t know what the data is. So you can’t improve it. I mean, it’s just completely impossible to do science with a model like this,” she says.
The mystery about how GPT-4 was trained is also a concern for van Dis’s colleague at Amsterdam, a psychologist Claudi Bockting. “It’s very hard as a human being to be accountable for something that you cannot oversee,” she says. “One of the concerns is they could be far more biased than for instance, the bias that human beings have by themselves.” Without being able to access the code behind GPT-4 it is impossible to see where the bias might have originated, or to remedy it, Luccioni explains.
Bockting and van Dis are also concerned that increasingly these AI systems are owned by big tech companies. They want to make sure the technology is properly tested and verified by scientists. “This is also an opportunity because collaboration with big tech can of course, speed up processes,” she adds.
Van Dis, Bockting and colleagues argued earlier this year for an urgent need to develop a set of ‘living’ guidelines to govern how AI and tools such as GPT-4 are used and developed. They are concerned that any legislation around AI technologies will struggle to keep up with the pace of development. Bockting and van Dis have convened an invitational summit at the University of Amsterdam on 11 April to discuss these concerns, with representatives from organizations including UNESCO’s science-ethics committee, Organisation for Economic Co-operation and Development and the World Economic Forum.
Despite the concern, GPT-4 and its future iterations will shake up science, says White. “I think it’s actually going to be a huge infrastructure change in science, almost like the internet was a big change,” he says. It won’t replace scientists, he adds, but could help with some tasks. “I think we’re going to start realizing we can connect papers, data programmes, libraries that we use and computational work or even robotic experiments.”
doi: https://doi.org/10.1038/d41586-023-00816-5
Originally published at https://www.nature.com on March 16, 2023.
Names mentioned (selected list)
- Evi-Anne van Dis — A psychologist at the University of Amsterdam who has seen demos of GPT-4.
- Sasha Luccioni — A research scientist specializing in climate at HuggingFace, an open-source-AI community, who is frustrated with OpenAI’s secrecy around GPT-4’s training data and code.
- Andrew White — A chemical engineer at the University of Rochester who has had privileged access to GPT-4 as a ‘red-teamer’.
- Claudi Bockting — A psychologist at the University of Amsterdam who is concerned about the biases in AI systems and the need for guidelines to govern their development and use.
