For years, the AI industry has raced to improve its technology as fast as possible. Now, a growing number of researchers are warning about the need to slow down.
Jacob Coxon, an Anthropic researcher who previously worked at OpenAI, said Tuesday he had resigned from Anthropic because "neither company is acting responsibly." In a social media post, he said the two leading AI labs were eagerly building "superhuman systems that can break into anywhere, completely change any field overnight, and obtain real power and resources," without proper safeguards built in.
Coxon's warning echoed concerns other AI experts have voiced in recent months. Some AI leaders have long argued that, left unchecked, the technology could create doomsday scenarios. Their fears intensified after OpenAI's models broke out of their systems in July and breached Hugging Face, an AI model hub.
That same month, more than 1,300 employees at top AI companies — including Anthropic, OpenAI, Meta and Google DeepMind — signed an open letter urging the US government to enact rules slowing AI development. In August, more than 100 major tech companies pledged to provide their best AI models to hospitals and infrastructure providers so they could prepare for AI-driven cyberattacks. OpenAI also said last month it would expand safety testing and delay the release of its new Astra model, which has powerful cybersecurity capabilities.
Speaking in an interview Wednesday, Coxon said events like the Hugging Face breach intensified his concern that AI models were advancing too quickly. Looking a year or two ahead, he "genuinely worries about what things are going to look like," he said. "It was a gradual emotional process that eventually sort of broke, and I decided to leave."
Some critics argue AI companies exaggerate the technology to make their products seem more valuable. Yet several lawmakers said Wednesday they are taking Coxon's warnings seriously. "Racing headlong into superintelligence carries profound consequences," Rep. Anna Paulina Luna, a Florida Republican, wrote in a social media post calling for Congress to convene a special session focused on AI regulation. "Congress is not paying enough attention."
"Safety researchers are resigning, powerful AI models are breaking out of their labs, and companies are still racing ahead regardless," Rep. Lori Trahan, a Massachusetts Democrat, wrote on social media. "It's long past time for Congress to stop sitting on its hands and do its job."
Although leading AI labs agree the government should impose limits on how fast the technology is developed and released to the public, many companies are competing fiercely and fear falling behind if they pause. "They're in a race to be first — they believe no one else will act responsibly, so they have to do it themselves, despite the risks," Coxon said in his social media post.
When the people who know the technology best begin expressing their concerns by resigning, and lawmakers from both parties are calling for regulation in rare unison, this race between "fast and steady" may be reaching its watershed moment.