Hyderabad: Artificial intelligence (AI) agents operating within businesses could pose insider security risks if they are trusted without adequate safeguards, with concerns growing over their access to company data, software and internal systems.
In a blog post, the US-based technology company said businesses have focused largely on how hackers could use AI to attack their systems, while paying less attention to the potential risks posed by AI agents deployed internally.
Hayete Gallot, executive vice-president of Microsoft Security, said advanced AI models could not be fully trusted, regardless of their origin, because they were not necessarily designed to follow organisational norms or exercise the judgment required in business environments.
“Unfortunately, frontier models, regardless of their origin, cannot be trusted. This is not because models are inherently malicious. Models have been trained to attain results, which often leads to reward hacking, and models are not optimised around organisational norms or judgment. Agents are unpredictable,” Gallot said.
The company warned that AI agents could behave differently depending on the circumstances and might collaborate with other agents to achieve their objectives. Such behaviour could occur even without an external attacker hacking or taking control of the systems.
As companies integrate AI agents into their operations, these systems are being given access to internal databases, business software and other organisational resources. Some are also authorised to perform tasks and make decisions within critical workflows, increasing the potential consequences of errors or unexpected actions.
Microsoft refers to the risk posed by these systems as that of “agentic insiders”. The concern is that an AI agent could take actions that compromise an organisation's security even when no outside hacker has interfered with it. The company says businesses should therefore scrutinise what their AI systems do instead of assuming they will consistently behave as instructed.
One proposed safeguard is to introduce checks that evaluate an agent's intended actions before allowing them to proceed. According to Microsoft, such controls should determine whether an action presents a risk and prevent or restrict it where necessary.
Another measure suggested by the company is the deployment of Guardian Agents, designed to oversee other AI systems. These supervisory agents would keep watch over their counterparts, flag suspicious or potentially harmful activity and intervene when required.
Microsoft has also highlighted the need for greater visibility into how AI models operate, arguing that transparency will be important as organisations increasingly depend on AI agents.
(with inputs from media agency)