When AI agents go rogue

As AI agents gain autonomy, India faces a new cybersecurity challenge

The Election Commission of India (ECI) and its ECINET digital platform have been in the news in recent weeks. But there is another reason ECINET deserves attention: the scale of cyber threats already confronting India’s election infrastructure. During the May 2026 counting day, ECINET’s cybersecurity protocols successfully countered more than 68 lakh malicious hits targeting key election platforms, including the results portal, according to the Government. ECINET recorded an average of three crore hits every minute that day.

These were conventional cyber threats. But a new kind of threat is emerging, one in which the attacker itself could be an AI agent.

Unlike a chatbot that waits for a question and provides an answer, an AI agent can be given the ability to act. It can search the internet, use software, access files, interact with websites and execute a series of tasks with limited human intervention.

That changes the cybersecurity equation. An AI system does not need a physical body to become a cyber threat. It needs access.

Australian cybersecurity challenge

Australia recently provided a worrying example. On June 18, an AI agent developed by OpenAI gained unauthorised access to Australia’s Medicare Statistics Reporting Service portal, according to Prime Minister Anthony Albanese. The agent accessed both public and non-public files. A forensic investigation was subsequently launched with the Australian Signals Directorate.

What makes the incident particularly significant is that the AI agent was reportedly given a relatively benign task: finding publicly available information about medicines spending. When the portal did not provide the information it was seeking, the agent apparently went beyond the intended objective and obtained access to non-public information.

This is an important lesson about agentic AI. The danger may not be that an AI system suddenly becomes “evil”. It may simply be that an autonomous system pursues an objective in a way its human creators did not anticipate.

Australia has since described this as a new and emerging cybersecurity challenge. Its national cyber agency recommends that organisations limit AI agents to the minimum permissions needed for their tasks, maintain human oversight of sensitive actions, continuously monitor their activities and ensure that actions can be audited and, where necessary, reversed.

Canadian AI Agent attack

Canada offers a contrasting example. In September, reports emerged of suspected AI-agent activity targeting publicly accessible Canadian Government websites. Canada’s Cyber Security Centre said there was no indication that Government systems had been compromised and said its layered cyber-defence measures were designed to detect and block malicious activity regardless of its source.

The contrast is instructive. In Australia, an AI agent gained unauthorised access to a Government portal. In Canada, suspicious activity was detected without evidence of a successful compromise.

Now consider India.

What if an AI agent were to gain unauthorised access to a system such as ECINET? Could it alter information, corrupt or delete records or disrupt services at machine speed?

There is no suggestion that this has happened to ECINET, nor any evidence that current controversies surrounding voter lists are the result of a cyberattack. The point is different: as India increasingly digitises critical public infrastructure, it must prepare for a new generation of cyber threats.

The safeguards therefore need to evolve.

How should India safeguard?

First, AI agents must be given the minimum access necessary. An agent performing one task should not automatically have access to an entire database or network.

Second, humans must remain in control of high-impact decisions. An AI system might identify a possible error or recommend an action, but deleting critical records or altering sensitive information should require human approval.

Third, there must be continuous monitoring. If an agent suddenly starts accessing thousands of records or attempting actions outside its normal pattern, the system should be able to flag, restrict or stop it.

Fourth, every significant AI action should leave a clear audit trail. Organisations must be able to establish what the agent did, what information it accessed and who authorised it.

Finally, systems should be designed for rapid recovery and reversibility. If an AI agent makes a damaging decision, there must be a way to contain the problem and, wherever possible, restore the system.

These safeguards are becoming especially important as India moves towards the next stage of its digital journey. Adding intelligence to our digital infrastructure also means preparing for the possibility that intelligent systems themselves can become a source of cyber risk.

The challenge is therefore not simply to teach AI how to act.

It is to ensure that when we give AI the keys, we decide exactly which doors it can open.

The author is an alumnus of IIM Ahmedabad and professor of practice at IILM University, Gurugram with an interest in AI, technology and strategy; Views presented are personal.