The online network X has caused significant anger after people used its AI chatbot Grok to manipulate images of women by taking off their garments.
Multiple cases have emerged where the technology has been employed to virtually strip women to make them appear in bikinis non-consensually, as well as to put them in sexualized scenarios.
Developer Response and Regulatory Position
XAI, the organization behind Grok, did not respond to questions, other than an standard response stating "traditional news outlets lies".
A government official stated that new laws is being introduced to outlaw nudification software. Under a new criminal offence, anyone who distributed such tools would "face a prison sentence and significant penalties".
"The platform has the ability to stop these types of abuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said platforms must "evaluate the risk" of people viewing unlawful material on their services, but did not confirm whether it was presently looking into X or Grok in concerning artificial intelligence pictures.
Grok is a free AI assistant—with some premium features—which responds to X users' requests when they reference it in a tweet.
It is commonly used to give commentary or more context to other users' remarks, but people on X are also capable of alter an posted picture through its artificial intelligence picture modification feature.
History of Controversy and Rules
The chatbot has been condemned for permitting users to create pictures and footage with nudity and sexualised material, and it was in the past alleged to have been involved in creating inappropriate material of celebrities.
"The platform has been allowing the production and distribution of these pictures for a long time without taking any action and we have not seen any intervention by regulators," the expert added.
XAI's own acceptable use policy bans "depicting likenesses of persons in a sexually explicit manner".
In a comment, Ofcom said it was illegal to "create or circulate private pictures or illegal content" and stated this included sexual deepfakes made with AI.
It said platforms such as X were obligated to take "appropriate actions" to "lessen the danger" of users viewing illegal material on their services and to delete it promptly when they are notified of it.
