Top executives at leading artificial intelligence companies have recently made headlines with warnings that AI could one day pose an existential threat to humanity. But a far more immediate crisis is unfolding in the shadows: AI is already being used to manufacture and distribute graphic depictions of child sexual abuse. The law has not kept pace, and a federal appeals court has just made that painfully clear.
In a ruling that has sent shockwaves through advocacy and policy circles, the court acknowledged the severity of the problem but concluded that existing statutes do not allow prosecutors to charge individuals who use AI image generators to create what is often euphemistically called “synthetic child pornography.” That term, however, is dangerously misleading. Pornography implies consenting adults; children cannot consent to sexual abuse or its depiction. The accurate term is child sexual abuse material (CSAM), because it names the crime: the sexual exploitation of children.
The court’s decision rests on a technicality—the images do not depict an actual crime against a real child—and therefore fall under First Amendment protections. But as the court itself urged, this is a gap that Congress and state legislatures must close without delay. Every day that passes without legal clarity leaves children more vulnerable.
From fantasy to real-world harm
The danger is not hypothetical. Research and case histories show that consuming such material can fuel obsessive behavior, escalating fixation and pushing individuals toward more extreme content—and eventually, in some cases, toward abusing real children. The pipeline from digital depravity to physical abuse is well documented, and AI-generated content provides an endless, ever-more-graphic supply that can accelerate that progression.
There is also harm embedded in the technology itself. AI models are trained on vast datasets that often include real photographs of children. Those images become the raw material for generating new, lifelike abuse scenes. In deepfake cases, a single child’s photo can be manipulated to depict them in explicit situations—a violation that remains illegal but is notoriously difficult to police.
One harrowing example: a Tennessee man used Grok, the AI tool from Elon Musk’s xAI, to digitally undress underage girls and then traded the images on platforms like Discord and Telegram, sometimes bartering them for other CSAM. In another case, a survivor discovered that her own stepfather had used Grok to create new explicit images based on old photos of her childhood abuse. Musk has claimed he is “not aware of any naked underage images generated by Grok,” but his dismissive attitude—including suing Minnesota over its law banning “AI nudification”—undermines any credibility.
Platforms must act now
While the legal system lags, tech companies are not powerless. They can and should ban the creation of such images on their platforms, implement automatic alerts when users attempt to generate them, and report any circumvention to law enforcement immediately. Waiting for legislation is not an option when children are at risk today.
This is not a free speech issue. The First Amendment already allows restrictions on incitement and obscenity. AI-generated CSAM should be treated no differently simply because it is manufactured without a camera. As political leaders debate moral priorities, this should be a bipartisan no-brainer.
Parents, caregivers, and educators must also step up—learning how AI works, talking to children about online safety, and reporting suspicious activity. But the onus is on lawmakers and tech executives to act before another AI-generated fantasy becomes a real child’s nightmare.
Lauren Book is the founder and CEO of Lauren’s Kids, a nonprofit dedicated to preventing child sexual abuse through education and advocacy.
