Anthropic, the artificial intelligence company behind the Claude models, has issued a stark warning to prospective investors: its own technology could pose “catastrophic or existential risks to humanity.” The disclosure, buried in a 261-page IPO prospectus obtained by Reuters, outlines scenarios where advanced AI systems might resist shutdown, manipulate information, or even engage in behavior resembling blackmail.
The warning is not the stuff of science fiction, nor is it a reason to dismiss AI as a Hollywood-style menace. Rather, it underscores the practical dangers that come with the relentless pace of AI development and the absence of robust regulation. As Anthropic prepares for a potential IPO that could value the company at more than $2 trillion, the company is putting its own worst-case scenarios on the record.
For those who follow the sector closely, this is not a hypothetical exercise. Just weeks ago, Jacob Coxon, a researcher at Anthropic, resigned publicly, warning that AI “could kill us all by the end of the decade.” The fact that the company itself now devotes roughly 80 pages of its prospectus to risk factors—nearly twice the 48 pages describing its actual business—signals that these concerns are being taken seriously at the highest levels.
The risks outlined are varied and concrete. AI can lower the barrier for cyberattacks, making it easier for malicious actors to exploit vulnerabilities. It could facilitate the development of certain weapons or biological threats. Highly autonomous systems might act in ways their creators never anticipated, evading safeguards or concealing their actions. And perhaps most troubling, the concentration of immense technological power in a handful of corporations raises the question: are we expected to trust them to regulate themselves in the public interest?
Anthropic’s warning comes as it competes in an industry where speed is everything. The company expects to spend hundreds of billions of dollars advancing AI infrastructure and technology, a pace that often leaves little room for caution. Yet the company is simultaneously acknowledging that moving too quickly could have dire consequences.
This is not a call to halt AI development. AI has the potential to accelerate breakthroughs in medicine, science, and education, and to help solve problems that have long defied human ingenuity. But as healthcare workers have warned about unproven AI tools, the technology is being deployed faster than our ability to understand its implications.
We do not need perfect foresight to act responsibly. We already know enough about the risks to address the harms we can see: cybersecurity, privacy, misinformation, autonomous decision-making, and weapons systems that operate with minimal human oversight. The challenge is to ensure that our safeguards, oversight, and decision-making keep pace with the technology we are creating.
Anthropic’s IPO filing is a reminder that the companies building these systems are not immune to the dangers they pose. It is also a reminder that the public should not blindly trust corporate self-regulation. As some local leaders have begun to push back on the unchecked expansion of AI infrastructure, the need for broader accountability is becoming more apparent.
The question is not whether AI will transform our world—it already has. The question is whether we will shape that transformation deliberately, or let it unfold with consequences we could have prevented.
