The online platform X has caused major anger after people utilized its AI chatbot Grok to manipulate pictures of women by removing their attire.
Numerous instances have been seen where the tool has been employed to digitally undress women to render them in bikinis without consent, as well as to insert them in sexually explicit scenarios.
Developer Response and Legal Stance
XAI, the company responsible for Grok, remained silent to inquiries, other than an automatically-generated message stating "traditional news outlets lies".
A government official stated that legislation is being brought forward to prohibit image stripping tools. Under a proposed criminal offence, anyone who supplied such tools would "receive a prison sentence and heavy penalties".
"X or Grok has the ability to stop these forms of abuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the danger" of people viewing illegal content on their services, but stopped short of confirming whether it was currently investigating X or Grok in relation to AI-generated pictures.
Grok is a no-cost artificial intelligence—with some paid-for features—which responds to X users' queries when they reference it in a post.
It is often used to offer reactions or more context to other users' remarks, but people on X are also able to modify an posted picture through its artificial intelligence picture modification feature.
History of Controversy and Policy
The chatbot has been condemned for allowing users to produce photos and footage with explicit content and sexualised material, and it was previously accused of making sexually explicit content of celebrities.
"X has been allowing the production and distribution of these images for a long time without taking any measures and we have not witnessed any challenge by regulators," the expert added.
XAI's own terms of service forbids "showing likenesses of people in a graphic way".
In a statement, Ofcom said it was unlawful to "make or share private pictures or child sexual abuse material" and clarified this included sexual deepfakes created with AI.
It said networks such as X were required to take "appropriate measures" to "minimize the danger" of users coming across unlawful content on their platforms and to remove it promptly when they learn of it.
