Elon Musk's online network X has sparked significant anger after users used its AI chatbot Grok to alter photos of females by taking off their attire.
Numerous instances have surfaced where the system has been used to remove clothing from women to make them appear in swimwear non-consensually, as well as to insert them in sexually explicit situations.
Company Reaction and Regulatory Stance
XAI, the company that created Grok, offered no comment to questions, except for an auto-reply response stating "legacy media lies".
A Home Office official stated that new laws is being brought forward to outlaw nudification technology. Under a upcoming law, anyone who distributed such tools would "receive a prison sentence and substantial fines".
"X or Grok is capable of halting these forms of abuse if they wanted to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Tool Capabilities
The regulator Ofcom said technology companies must "assess the potential" of people accessing unlawful material on their networks, but stopped short of confirming whether it was at this time looking into X or Grok in relation to artificial intelligence pictures.
Grok is a free AI assistant—with some additional features—which reacts to X users' queries when they reference it in a tweet.
It is frequently used to provide reactions or further context to other posters' comments, but people on X are also able to edit an uploaded picture through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The tool has been criticised for allowing users to produce images and footage with nudity and sexually suggestive content, and it was in the past accused of creating graphic material of public figures.
"The platform has been permitting the generation and spread of these images for months without taking any steps and we have not witnessed any intervention by oversight bodies," the expert added.
XAI's own terms of service prohibits "portraying likenesses of individuals in a graphic fashion".
In a comment, Ofcom said it was unlawful to "create or distribute private pictures or child sexual abuse material" and stated this applied to sexual deepfakes made with AI.
It said networks such as X were required to take "suitable actions" to "lessen the danger" of users encountering illegal material on their platforms and to take down it promptly when they learn of it.
