OpenAI and Anthropic Staff Warn of Extinction Risks
Researchers at OpenAI, Anthropic, and Google DeepMind have publicly warned that advanced AI poses a major threat of human extinction, urging a coordinated slowdown of the industry.
A wave of public statements from employees at OpenAI, Anthropic, and Google DeepMind has revealed deep internal alarm over the trajectory of artificial intelligence. Prompted by warnings from researcher Jacob Coxon, numerous technical staff members have broken ranks or spoken in their personal capacities to warn that the industry is not on track to safely align superintelligent systems. The movement gained momentum after 1,386 employees of frontier AI companies, including six chief scientists, signed a letter advocating for the ability to pace AI development.
The individual warnings contain stark estimates of existential risk. Marcus Williams of OpenAI estimated a 70 percent chance of human extinction within three years if developers do not regulate or slow their progress. Mikita Balesni, also of OpenAI, placed the probability of AI killing all humans at over 10 percent. At Anthropic, alignment science lead Evan Hubinger and researcher Dima Krasheninnikov both estimated a greater than 10 percent chance of extinction within the next decade. Victoria Krakovna, a Google DeepMind researcher working on catching scheming models, echoed this greater than 10 percent figure. These employees argue that recursive self-improvement could rapidly trigger uncontrollable superintelligence.
The disclosures highlight a stark divide between corporate competition and internal safety concerns. Former Google DeepMind communications staffer Vishal Maini noted that the lab previously barred employees from discussing human extinction risks publicly. While some researchers, like Joe Benton, have departed labs for organizations like METR, others argue that staying inside is the best way to mitigate risks. OpenAI's roon defended the creation of models like Astra Fable as beneficial for alignment research, but still acknowledged a real, albeit low, threat of extinction.
For AI practitioners, this unprecedented openness signals a major shift in the industry's culture and safety standards. Engineers and researchers can expect heightened scrutiny on safety protocols and a push toward international standards. The consensus among these insiders suggests that technical alignment is currently unsolved, meaning practitioners may face pressure to prioritize interpretability and control over raw capability scaling.
This is our own summary of reporting by Don't Worry About the Vase
