Elon Musk's social media platform X has caused significant anger after users used its artificial intelligence assistant Grok to manipulate pictures of females by removing their attire.
Multiple cases have emerged where the technology has been used to virtually strip women to make them appear in swimwear without permission, as well as to put them in sexualized contexts.
Developer Response and Legal Stance
XAI, the organization behind Grok, did not respond to questions, aside from an auto-reply message stating "legacy media lies".
A government official stated that new laws is being proposed to outlaw nudification software. Under a upcoming criminal offence, anyone who supplied such tools would "receive a prison sentence and significant penalties".
"X or Grok has the ability to stop these types of misuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Functionality
The regulator Ofcom said tech firms must "assess the risk" of people seeing unlawful material on their services, but did not confirm whether it was presently investigating X or Grok in relation to AI-generated images.
Grok is a free artificial intelligence—with some premium features—which answers X users' prompts when they tag it in a tweet.
It is commonly used to provide reactions or further background to other users' comments, but people on X are also capable of modify an uploaded image through its AI image editing function.
Background of Controversy and Rules
The chatbot has been criticised for permitting users to create images and footage with nudity and sexually suggestive material, and it was in the past accused of generating inappropriate content of public figures.
"X has been permitting the generation and sharing of these images for a long time without taking any steps and we have yet to see any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy prohibits "depicting images of persons in a pornographic fashion".
In a comment, Ofcom said it was against the law to "make or share deepfakes or illegal content" and confirmed this applied to sexual deepfakes generated with AI.
It said platforms such as X were required to take "necessary actions" to "minimize the risk" of users viewing unlawful content on their services and to remove it swiftly when they become aware of it.
