Elon Musk's social media platform X has sparked significant backlash after users used its AI chatbot Grok to alter pictures of women by removing their attire.
Numerous instances have emerged where the system has been used to remove clothing from women to make them appear in swimwear without consent, as well as to place them in sexualized scenarios.
Developer Reaction and Regulatory Stance
XAI, the firm that created Grok, offered no comment to inquiries, other than an automatically-generated response stating "traditional news outlets lies".
A government official stated that new laws is being brought forward to outlaw nudification technology. Under a upcoming criminal offence, anyone who distributed such tools would "face a jail term and substantial fines".
"The platform has the ability to stop these forms of misuse if they desired to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said technology companies must "evaluate the danger" of people viewing illegal content on their services, but did not specify whether it was currently looking into X or Grok in concerning AI-generated pictures.
Grok is a free AI assistant—with some additional functions—which answers X users' queries when they tag it in a post.
It is often used to give commentary or additional background to other users' remarks, but people on X are also capable of modify an posted image through its AI image editing feature.
Background of Controversy and Rules
The tool has been faulted for permitting users to produce photos and clips with explicit content and sexualised material, and it was previously alleged to have been involved in making sexually explicit content of public figures.
"X has been allowing the creation and sharing of these pictures for months without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "depicting images of persons in a sexually explicit manner".
In a statement, Ofcom said it was unlawful to "create or share private pictures or child sexual abuse material" and stated this included sexual deepfakes created with AI.
It said networks such as X were mandated to take "appropriate steps" to "minimize the danger" of users viewing illegal content on their services and to delete it quickly when they become aware of it.
