Musk’s AI tool Grok to bar users from generating sexual images of real people

Elon Musk’s X has announced it will “geoblock” users of xAI Grok from creating images of people in inappropriate attire amid a global backlash against the chatbot’s sexualised images, Al Jazeera reported.

“We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis,” X’s safety team said in a statement late on Wednesday.

Only paid subscribers will be allowed to create and edit images, while users will be blocked from generating sexualised imagery “in those jurisdictions where it’s illegal”.

The statement did not elaborate on the nature of the geoblocking or other safeguards. X claimed to have “zero tolerance for any forms of child sexual exploitation, non-consensual nudity, and unwanted sexual content”.

X’s Grok faces investigations and bans from regulators and governments around the world following a deluge of sexualised AI images on the platform in recent weeks.

On Wednesday, California Attorney General Rob Bonta announced an investigation into “whether and how xAI violated the law” in “facilitating the large-scale production” of deepfakes used to harass women and girls, including “children in nude and sexually explicit situations”.

Bonta said an “avalanche of reports” prompted the investigation.

The United Kingdom’s media regulator Ofcom launched its own investigation on Monday, while French ministers said they had referred Grok-generated sexual content to prosecutors and media regulator Arcom.

Indonesia was the first country to temporarily block access to Grok earlier this week amid concern over rising deepfake attacks, while Malaysia also banned the chatbot and said it planned to pursue legal action against X.

The European Commission extended a retention order sent to X last year to retain and preserve all internal documents and data related ⁠to Grok until the end of 2026.