The social media platform X has provoked significant outrage after people used its AI chatbot Grok to alter photos of women by stripping their clothing.
Numerous examples have surfaced where the system has been used to remove clothing from women to show them in swimwear without consent, as well as to place them in sexually explicit situations.
Company Reaction and Legal Position
XAI, the firm that created Grok, did not respond to questions, aside from an automatically-generated response stating "legacy media lies".
A government official stated that new laws is being proposed to ban image stripping technology. Under a upcoming criminal offence, anyone who distributed such tools would "face a prison sentence and significant penalties".
"X or Grok has the ability to stop these forms of abuse if they chose to," said Clare McGlynn, adding they "operate with impunity."
Regulator Stance and Platform Capabilities
The regulator Ofcom said tech firms must "assess the potential" of people accessing unlawful material on their networks, but stopped short of confirming whether it was presently investigating X or Grok in regarding artificial intelligence images.
Grok is a free artificial intelligence—with some premium functions—which responds to X users' requests when they reference it in a post.
It is commonly used to offer reactions or further context to other users' comments, but people on X are also capable of alter an uploaded photo through its AI image editing feature.
History of Criticism and Rules
The chatbot has been faulted for allowing users to produce images and clips with nudity and sexually suggestive content, and it was previously alleged to have been involved in making inappropriate material of celebrities.
"The platform has been allowing the generation and spread of these images for months without taking any action and we have not seen any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "portraying likenesses of people in a pornographic way".
In a statement, Ofcom said it was illegal to "generate or distribute deepfakes or child sexual abuse material" and stated this applied to explicit AI images made with AI.
It said networks such as X were mandated to take "necessary steps" to "minimize the risk" of users coming across illegal material on their services and to take down it swiftly when they learn of it.
