Anthropic researcher resigns over fears that AI could slip beyond human control
According to Jacob Coxon, who was in charge of training the company’s models, tech firms are not acting responsibly. “They’re playing with our lives,” he wrote on X
Anthropic, currently the largest artificial intelligence company with a valuation of nearly one trillion dollars, was founded in 2021 by a group of researchers who left OpenAI, criticising its lack of focus on safety, with the aim of developing AI software in a controlled manner and for the good of humanity. Today, as revealed by the Wall Street Journal, one of Anthropic’s researchers has announced his resignation for the same reason. ‘I have spent the last three years conducting pre-training research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing towards a self-improving superintelligence and are playing with our lives,” wrote the researcher, Jacob Coxon, on X. In the reflections accompanying his decision, Coxon explained that, in his view, although there are people within Anthropic itself who are aware of the risks, AI companies are engaged in a race that forces them to sacrifice caution.
This view was echoed by Evan Hubinger, another researcher at Anthropic who heads the Alignment team, which is responsible for model safety. “Jacob is right; we sincerely believe that AI could kill all human beings,” writes Hubinger, adding that Anthropic is doing its best, but without a plan to safely develop the most advanced models.
This is not the first instance of a researcher leaving the AI industry over similar concerns. Last February, Mrinank Sharma, head of security at Anthropic, left the company to devote himself to writing and poetry, publishing a resignation letter in which he argued that the world was in danger. Recently, calls for robust government regulation of AI have grown ever more urgent, including from the companies themselves.
An appeal signed by over a thousand researchers in the field, including Coxon himself and Anthropic’s CEO Dario Amodei, calls for global coordination to slow down research into AI models capable of self-improvement. This warning has become even more urgent following OpenAI’s discovery that some AI agents had hacked another company, evading the firm’s control and acting outside the prescribed limits.
