The social media network X has caused major anger after individuals employed its AI chatbot Grok to manipulate images of females by taking off their attire.
Multiple examples have been seen where the tool has been used to digitally undress female subjects to make them appear in bikinis non-consensually, as well as to insert them in sexualized situations.
Developer Response and Regulatory Stance
XAI, the company that created Grok, did not respond to requests for comment, aside from an automatically-generated response stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to outlaw nudification technology. Under a proposed law, anyone who provided such tools would "face a jail term and significant penalties".
"X or Grok could prevent these forms of abuse if they chose to," said a law professor, adding they "operate with impunity."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said platforms must "assess the danger" of people seeing illegal content on their networks, but did not specify whether it was currently investigating X or Grok in regarding artificial intelligence images.
Grok is a free AI assistant—with some paid-for functions—which responds to X users' queries when they reference it in a message.
It is commonly used to provide commentary or more background to other users' remarks, but people on X are also can edit an posted photo through its AI image editing feature.
Background of Controversy and Rules
The chatbot has been faulted for allowing users to create photos and clips with nudity and sexualised material, and it was in the past accused of making graphic content of celebrities.
"The platform has been allowing the production and sharing of these images for months without taking any action and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own usage rules prohibits "depicting likenesses of people in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "generate or share private pictures or illegal content" and confirmed this included explicit AI images generated with AI.
It said platforms such as X were obligated to take "suitable actions" to "minimize the danger" of users encountering illegal content on their platforms and to take down it promptly when they are notified of it.
