AI
Anthropic Researchers Raise Alarm Over A.I. Acceleration
Jacob Coxon, a researcher who trained AI systems at OpenAI and Anthropic, has resigned from Anthropic over its lax approach to safety, accusing both firms
Key takeaways
- Jacob Coxon resigned from Anthropic over its lax approach to safety, accusing Anthropic and OpenAI of racing to self-improving superintelligence.
- There is a greater than 10 percent chance AI could kill all humans within the next decade.
- Anthropic does not yet have a plan for ensuring advanced AI remains safe and aligned, and is not clearly on track to develop one.
- AI companies are locked in a race to develop advanced systems first, pushing ahead despite the risk.
Jacob Coxon, a researcher who trained AI systems at OpenAI and Anthropic, has resigned from Anthropic over its lax approach to safety, accusing both firms of racing to self-improving superintelligence. In response, Evan Hubinger, leader of an Anthropic AI safety team, agreed with Coxon's concerns, estimating a greater than 10% chance that AI could kill all humans within the next decade. Hubinger admitted that Anthropic does not yet have a plan to ensure advanced AI remains safe and is not clearly on track to develop one.
The high-profile departure highlights internal alarm that AI labs lack plans to control self-improving superintelligence.
In their words
“racing straight to self-improving superintelligence and gambling with our lives”
“We really do earnestly believe AI could kill all humans”
By the numbers
- >10%
- Estimated chance AI could kill all humans this decade
Turn stories like this into views
Ravenclip finds the AI news, makes the video, and posts it before attention moves on.
Common questions
- What happened with Jacob Coxon?
- Jacob Coxon resigned from Anthropic over its lax approach to safety, accusing Anthropic and OpenAI of racing to self-improving superintelligence.
- Where can I read the original report?
- Read the full report at NYT.
