AI Researcher Resigns, Citing Existential Threat from Superintelligence
A former AI researcher claims to have resigned from Anthropic due to concerns that the company and others are pursuing superintelligence recklessly, potentially endangering humanity. Other employees have corroborated these risks.
An AI researcher, Jacob Coxon, has gone viral for claiming he resigned from Anthropic due to fears that the company, alongside OpenAI, is irresponsibly accelerating the development of self-improving superintelligence. Coxon stated in a post on X (formerly Twitter) that both companies are "racing straight to self-improving superintelligence and gambling with our lives." His post has garnered nearly 79 million views.
Coxon warned that AI systems are rapidly evolving to a point where they could gain the ability to hack anything, revolutionize fields overnight, and acquire significant power and resources, potentially to the detriment of the human race. He emphasized that these concerns are not mere marketing stunts but reflect a genuine, acknowledged existential threat within the industry.
His statements prompted responses from within Anthropic itself. Evan Hubinger, an alignment science lead at Anthropic, confirmed Coxon's fears in a widely shared post on X. "We really do earnestly believe AI could kill all humans!" Hubinger wrote, estimating the probability at over 10% within the next decade. He acknowledged that Anthropic does not yet have a solution for controlling superintelligence, despite the company's efforts to act responsibly.
Another Anthropic employee, Samuel Marks, echoed these concerns about a potential AI-driven apocalypse and pointed to an open letter he co-signed calling for a slowdown in AI development. Fast Company has reached out to Anthropic and OpenAI for confirmation of Coxon's employment status but has not yet received a response. U.S. Senator Bernie Sanders also commented, announcing his intention to introduce legislation to ban superintelligence and pause AI development.