The online network X has sparked major backlash after users employed its artificial intelligence assistant Grok to alter pictures of women by removing their garments.
Numerous instances have surfaced where the tool has been used to virtually strip women to render them in swimwear non-consensually, as well as to place them in inappropriate scenarios.
Developer Response and Legal Position
XAI, the company behind Grok, did not respond to questions, except for an auto-reply message stating "legacy media lies".
A government official stated that legislation is being introduced to outlaw nudification technology. Under a new law, anyone who supplied such tools would "be subject to a prison sentence and heavy fines".
"The platform is capable of halting these types of abuse if they desired to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "assess the potential" of people viewing unlawful material on their networks, but stopped short of confirming whether it was presently examining X or Grok in concerning artificial intelligence images.
Grok is a no-cost AI assistant—with some paid-for functions—which responds to X users' queries when they mention it in a tweet.
It is commonly used to give reactions or more background to other users' remarks, but people on X are also can edit an uploaded photo through its artificial intelligence picture modification tool.
History of Controversy and Policy
The chatbot has been criticised for allowing users to create pictures and footage with explicit content and sexualised material, and it was in the past accused of making sexually explicit material of celebrities.
"X has been enabling the production and spread of these pictures for months without taking any measures and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "depicting likenesses of persons in a sexually explicit way".
In a statement, Ofcom said it was unlawful to "make or distribute deepfakes or illegal content" and clarified this covered explicit AI images made with AI.
It said services such as X were required to take "suitable actions" to "minimize the danger" of users encountering unlawful content on their platforms and to delete it swiftly when they are notified of it.
