X begins acting on obscene AI content

Social media platform X, formerly Twitter, admitted its mistake and removed about 3,500 pieces of content, as well as deleted over 600 accounts, after the Information Technology (IT)Ministry raised concerns over obscene content linked to Elon Musk-backed AI chatbot Grok, Government sources said on Sunday. X has assured the authorities that it would comply with Indian laws.

The action comes amid mounting pressure on Grok from Governments worldwide, as regulators intensify scrutiny of the generative AI engine over content moderation, data safety and non-consensual sexually-explicit images that have flooded X over the last few days.

According to Indian Government sources, X has accepted its mistake and assured it will comply with Indian laws. Accordingly, it has blocked around 3,500 pieces of content, and over 600 accounts have been deleted. It has also assured that going forward, it won’t allow obscene imagery, sources said.

Last Sunday, X’s ‘Safety’ handle said it takes action against illegal content on its platform, including Child Sexual Abuse Material (CSAM), by removing it, permanently suspending accounts, and working with local Governments and law enforcement as necessary.

“Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content,” X had said, echoing the stance taken by Musk on illegal content.

X is facing a global backlash over the ‘digital undressing’ of images through user prompts on the xAI chatbot Grok. It has come under fire from Governments across the world, including in Europe and Asia, warning it of consequences if the platform failed to adhere to online safety norms, while public calls for action on both X and Grok over the issue have been growing.

On January 2, the IT Ministry had pulled up X and directed it to immediately remove all vulgar, obscene and unlawful content generated by Grok or face action under the law.

In the directive, the ministry asked the US-based social media firm to submit a detailed action taken report (ATR) within 72 hours, spelling out specific technical and organisational measures adopted or proposed in relation to the Grok application, the role and oversight exercised by the Chief Compliance Officer, actions taken against offending content, users and accounts, as well as mechanisms to ensure compliance with the mandatory reporting requirement under Indian laws.

The IT Ministry’s missive had noted that Grok AI, developed by X and integrated on the platform, is being misused by users to create fake accounts to host, generate, publish or share obscene images or videos of women in a derogatory or vulgar manner.

“Importantly, this is not limited to creation of fake accounts but also targets women who host or publish their images or videos, through prompts, image manipulation and synthetic outputs,” the ministry had said, asserting that such conduct reflects a serious failure of platform-level safeguards and enforcement mechanisms, and amounts to gross misuse of artificial intelligence (AI) technologies in violation of stipulated laws.

The Government had made it clear to X that compliance with the IT Act and rules is not optional, and that the statutory exemptions under section 79 of the IT Act (which deals with safe harbour and immunity from liability for online intermediaries) are conditional upon strict observance of due diligence obligations.

The Government warned X in clear terms that any failure to observe due diligence obligations will result in the loss of the exemption from liability (under section 79 of the IT Act), and that the platform will also be liable for consequential action under other laws, including the IT Act and Bharatiya Nyaya Sanhita.It asked X to enforce user terms of service and AI usage restrictions, including ensuring strong deterrent measures, such as suspension, termination and other enforcement actions against violating users and accounts.

X was also asked to remove or disable access “without delay” to all content already generated or disseminated in violation of applicable laws, in strict compliance with the timelines prescribed under the IT Rules, 2021, without, as such, vitiating the evidence.

The initial response by X to the notice, though detailed, was seen as a mere reiteration by the platform that it respects Indian laws and stipulated guidelines, and that India is a big market for the platform.

In its response, X also outlined the strict content takedown policies it abides by when it comes to misleading posts and those related to non-consensual sexualised images.Although that reply was long and detailed, the IT Ministry was of the opinion that it “missed” key information, including takedown details and specific action that had been taken on the Grok AI obscene content issue, and to prevent it in future.

The response fell short of expectations, prompting the Government to toughen its stance and ask for more details, including specific action taken on obscene content linked to its Grok AI and measures to prevent a repeat in future.X has now accepted its mistake and told the Government that it will comply with Indian laws and regulations, sources said.Apart from India, the UK and the EU have also mounted pressure on X over the Grok deepfake image issue. British Prime Minister Keir Starmer termed it “disgusting”, while the European Commission has reportedly asked X to retain all documents relating to the AI chatbot Grok for longer.

Ofcom, the UK’s independent communications regulator, in a recent social media post, said: “We are aware of serious concerns raised about a feature on Grok on X that produces undressed images of people and sexualised images of children”.“We have made urgent contact with X and xAI to understand what steps they have taken to comply with their legal duties to protect users in the UK. Based on their response, we will undertake a swift assessment to determine whether there are potential compliance issues that warrant investigation,” Ofcom had said.

During the week, three Democratic US senators, in an open letter, called on Apple and Google to suspend X and Grok from their app stores for violating terms of service.