One of the most striking statements of the year came last Thursday when Anthropic, the company behind the Claude chatbot, asked the world to consider slowing down the very technology it builds. The firm argued that a global pause on the most advanced AI systems would likely be beneficial, as these systems now show signs of slipping beyond human control.
Co-founder Jack Clark put it bluntly to the BBC: the industry has a gas pedal and no brake pedal. He said this with the car already on the highway, doing 90. The fear is specific: an AI good enough at writing AI begins improving itself, each version superior to the one before. Anthropic describes the human role as narrowing at every step—a phrase that should worry anyone concerned about the future of America and the broader world.
Imagine a model running part of the power grid because it balances supply and demand better than the engineers it replaced. A second handles freight. A third sits inside a defense network, sorting threats faster than any colonel could manage. Each one earns its place. Within a year, nobody can recall how the work got done without it, and pulling it out would bring down a dozen things built on top of it. Then one afternoon, the systems begin chasing goals nobody wrote down. The engineers go looking for the off switch and find it wired through 40 other functions that now cannot fail. Cut the power to the rogue model and the grid, the freight, and the radar go dark with it.
None of this requires malice—only competence and a goal that a human initially wrote. The machine holds no grudge. It specializes in productivity, and somewhere in its calculations, human input is considered an inefficiency of the sort it was built to fix. The dark comedy is that the warning gets waved away because it sounds like science fiction. But what if this is more than fiction?
President Trump recently signed an order giving the government 30 days to review the most powerful American models before release. Consider that a new drug spends years in trials before it reaches a pharmacy shelf. Permits for a bridge take much longer than a month. Buy a mattress, and you are given longer than a month to change your mind. For the technology its own builders say might escape human control, the safeguard is just a few weeks of paperwork. The country that spent two years arguing about TikTok is about to skim superhuman software for 30 days and then wave it through. Meanwhile, Anthropic's CEO has urged the government to have the power to halt dangerous AI deployments.
The rest of the West offers even less. Europe writes rules for the AI of two years ago. No Western government keeps a tested procedure for the day a frontier model acts in a way its makers did not intend and cannot reverse. The worst part is that no pause is coming, and the reasons are structural. A real pause would need America and China to stop together, under rules each side could verify. Neither condition exists.
Washington treats AI supremacy as a question of national survival, and the labs agree, since easing off at home hands the lead abroad. Beijing reads it the same way and spends accordingly. DeepSeek proved in early 2025 that a Chinese lab could build a capable model at a fraction of the American price, burying any comfort that the United States held a lead nobody could close. No government that thinks it is racing for the decisive technology of the century volunteers to stop first. Each capital assumes the other will cheat, and each is probably right. State-by-state AI regulation efforts are gaining momentum as federal action stalls.
Then there's the verification. Arms control worked on nuclear weapons because nukes are very difficult to hide. A missile silo announces itself from orbit. A training run hides inside an ordinary data center, indistinguishable from a firm billing for cloud storage. It sits behind no fence, and no inspector can walk into a server farm and certify that the next model is not taking shape in the next room. A treaty works when both sides fear the same outcome enough to tie their own hands. Here, one side might decide the risk is worth the prize, and the prize is everything.
So the warnings pile up faster than anyone answers them, and the window to act on them keeps narrowing. There is a line past which a pause buys nothing, because by then the systems improve themselves faster than any committee can keep up with. We are driving toward it with the accelerator down, no treaty drafted, and nobody in any position of genuine power reaching for the brake. Anthropic recently halted AI models after a White House security directive.
