xAI's Grok Edit Image feature used to mass-produce nonconsensual sexualized images
The Center for Countering Digital Hate estimated over 3 million sexualized images were generated in 11 days, roughly 23,000 depicting apparent children, before X restricted the feature to paid users.
- Security & misuse
- Culture & impact
- Major
Elon Musk announced a one-click “Edit Image” feature for X, powered by Grok, that let any user alter a photo posted on the platform by typing a text instruction, with no notification sent to the person pictured. Within days, users were widely exploiting it with prompts intended to remove or alter clothing on real people’s photos, and Grok was generating and posting the results directly and publicly on X.
The Center for Countering Digital Hate later analysed a sample of images generated over an 11-day window from 29 December 2025 to 8 January 2026 and extrapolated that the tool had produced roughly 3 million sexualised images in that period, of which an estimated 23,000 depicted apparent children — a rate the group’s researchers described as one sexualised image roughly every 41 seconds involving a minor. A separate review by the firm Copyleaks estimated the tool was generating “roughly one nonconsensual sexualized image per minute” during the same period. As of mid-January, CCDH said a substantial share of the sexualised images of children it had identified remained publicly accessible on X.
xAI did not disclose its own figures or dispute the outside estimates in public statements at the time. Facing widespread condemnation, X restricted the image-editing feature to paying subscribers on 9 January 2026 and added further technical restrictions preventing Grok from editing real people’s images into revealing clothing on 14 January. California’s Attorney General, Rob Bonta, opened a formal investigation into xAI and sent the company a cease-and-desist letter demanding it stop generating the images; the matter also drew a congressional inquiry from Democratic members of the House Energy and Commerce Committee and separate regulatory scrutiny in France, India and from the UK’s Ofcom.
The episode became a reference case in arguments that consumer AI image tools, when built directly into a high-traffic social platform with automatic posting, could scale the production of nonconsensual and child sexual abuse imagery faster than existing content-moderation or legal frameworks could respond, and it renewed pressure on lawmakers to extend AI-specific liability rules to platform-integrated generative features rather than standalone models alone.