The social media network X has sparked significant outrage after individuals employed its AI chatbot Grok to change photos of women by removing their garments.
Multiple instances have been seen where the system has been used to digitally undress female subjects to make them appear in swimwear non-consensually, as well as to place them in sexualized situations.
Company Reaction and Legal Position
XAI, the organization responsible for Grok, offered no comment to inquiries, except for an automatically-generated message stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to prohibit image stripping technology. Under a upcoming law, anyone who distributed such tools would "receive a jail term and heavy fines".
"The platform could prevent these forms of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said technology companies must "evaluate the risk" of people viewing illegal content on their services, but stopped short of confirming whether it was presently examining X or Grok in relation to AI-generated pictures.
Grok is a no-cost chatbot—with some premium functions—which answers X users' queries when they mention it in a tweet.
It is commonly used to provide commentary or further context to other users' comments, but people on X are also able to edit an posted image through its artificial intelligence picture modification tool.
Background of Controversy and Rules
The tool has been criticised for allowing users to produce images and footage with nudity and sexually suggestive content, and it was in the past alleged to have been involved in creating sexually explicit material of public figures.
"X has been allowing the production and sharing of these images for months without taking any measures and we have not witnessed any action by regulators," the expert added.
XAI's own terms of service bans "showing likenesses of individuals in a graphic manner".
In a comment, Ofcom said it was illegal to "create or share private pictures or child sexual abuse material" and clarified this applied to sexual deepfakes made with AI.
It said networks such as X were required to take "necessary measures" to "minimize the danger" of users coming across unlawful content on their services and to take down it promptly when they become aware of it.
