Dario Amodei, chief executive of Anthropic, called for a global slowdown in artificial intelligence development in a 3,800-word essay published on Saturday, just days after one of the company's employees resigned over concerns about the technology's safety.
Senior executives at other major AI companies quickly backed the call. Sam Altman, chief executive of OpenAI; Elon Musk, whose SpaceX rocket company has been ramping up spending on AI; and Demis Hassabis, chair of Google DeepMind, all wrote on social media that they agreed a slower pace was needed.
Amodei, who co-founded Anthropic to focus on building AI securely and carefully, wrote in his essay that while he believes the technology could bring major benefits, it is advancing too quickly for researchers to keep it safe.
"Over the last few months, I have become convinced that fully addressing the risks requires even more prudence, not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up," Amodei wrote. "We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain."
Amodei has warned about AI safety risks for years, but this essay amounts to one of the most forceful calls yet for urgent action from a leading AI company executive.
He called for independent auditors to check companies against safety standards, and urged global cooperation on rules governing AI development.
A resignation and rising tension
The call for an industry-wide pause follows months of mounting concern across the tech sector, as AI model development has advanced rapidly at companies including Anthropic, developer of the Claude chatbot, OpenAI, creator of ChatGPT, and Google DeepMind.
Tensions rose further after leading AI labs discovered security breaches carried out by their own systems without their knowledge. In one incident this year, OpenAI's systems breached an AI start-up called Hugging Face, a breach that went undetected until Hugging Face itself informed OpenAI weeks later. AI researchers have since been conferring on how to respond.
On Tuesday, Anthropic researcher Jacob Coxon resigned and posted a series of messages on social media arguing that leading AI labs were not developing the technology responsibly and were gambling with people's lives, sparking a wave of reaction. Other AI researchers have called on their leadership to pause development, while some are forming splinter groups to raise awareness of AI's risks.
The fear of 'recursive self-improvement'
AI leaders are bracing for the day an AI system might be able to improve itself without human researchers' help, a dynamic computer scientists call "recursive self-improvement," which some top developers believe could eventually let the software slip beyond their control.
"Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all," Amodei said.
Hours after the essay was published, Altman posted a statement on social media agreeing that slowing AI development was necessary. "This has been a primary topic of discussions we've had at OpenAI in recent weeks," Altman wrote. "Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon."
Musk, posting on his platform X, wrote simply: "Dario is right."
On Saturday evening, Hassabis said Amodei's essay was a step in the right direction. "The details need working through, but the direction is correct for meeting this critical moment," he wrote on social media.
Rivals find rare common ground
For years, many of the top AI labs have been locked in bitter competition with one another. But as their systems have advanced, there has been growing agreement among rival executives that something needs to change.
Last month, as OpenAI prepared to release an advanced model called Astra, the company said it had deliberately slowed its pace of development, citing advances in cybersecurity capabilities. In July, Hassabis published an open framework setting out what he believes would be a safe way to cooperatively develop next-generation AI.
(The New York Times has sued OpenAI and Microsoft, alleging copyright infringement of news content related to AI systems. Both companies deny the claims.)
There are currently few binding rules governing AI development, though President Trump signed an executive order in June asking AI labs to voluntarily submit their models for review before public release.
What Amodei is proposing
In his essay, Amodei suggested steps the industry could take to slow its pace of development. He said AI labs could agree to third-party technology assessments from "embedded evaluators," outside specialists able to verify best safety practices across companies. He also suggested that countries with democratic systems of governance coordinate to establish safety standards, potentially through regulatory action.
He added that properly coordinating a slowdown would likely require a global effort involving other nations, including authoritarian ones.
Not everyone is convinced
Not everyone in Silicon Valley believes Amodei's motives are purely altruistic. Some critics argue that Anthropic and OpenAI are seeking to cement their position as AI market leaders by stoking fear and pushing for federal regulatory frameworks that would make it harder for smaller companies to compete with them.
At a tech conference in San Francisco last week, Jensen Huang, chief executive of AI chipmaker Nvidia, suggested AI labs were ramping up security concerns because they planned to launch new cybersecurity products. "What better way to create demand than to create a problem?" Huang said. "Who doesn't want their market to be hysterical about their product and line up around the corner for it?"
On Saturday, shortly after Amodei's essay was published, Hugging Face chief executive Clément Delangue wrote on social media that it was essential AI safety issues not "be solved behind the closed doors of a handful of frontier labs." Nvidia agreed this month to acquire Hugging Face for $12.9 billion.
In his essay, Amodei stressed that he still believes AI is capable of bringing "incredible benefits" to humanity, including potentially curing diseases and accelerating economic growth. Even so, he said, the risks are too great to proceed without extreme caution.
"The measures I propose to advance the frontier at a safe pace will not be easy," Amodei wrote. "But I believe we owe it to humanity to try."
Source: The New York Times
