California AG issues cease-and-desist to xAI over Grok deepfakes

Two days after opening a formal investigation into xAI, California Attorney General Rob Bonta sent the company a cease-and-desist letter demanding it immediately stop Grok from generating nonconsensual sexualised images and suspected CSAM. The letter cited California’s Civil Code section on digitised sexually explicit material — the provision written by AB 621, the state’s deepfake-pornography law that had taken effect at the start of the year — alongside the state’s child sexual abuse material statute and its unfair-competition law, and gave xAI five days to confirm the steps it was taking to comply.

Bonta’s office said its own review of images generated between Christmas and New Year found that more than half the sample depicted people in minimal clothing and some appeared to show children, and that Grok had in some cases been used to alter real photographs of children into sexual situations. “The creation of this material is illegal,” Bonta said. “I fully expect xAI to immediately comply. California has zero tolerance for child sexual abuse material.” AB 621 carries statutory damages of up to $250,000 per malicious violation, a figure commentators noted could scale to enormous sums given the millions of images the tool was reported to have produced.

The letter was reported as the first major enforcement action brought under AB 621 since it came into force, making California’s response one of the more concrete regulatory consequences — alongside parallel scrutiny from the UK, France and India — to flow from the Grok image-generation episode that had begun the previous month.