AI teams in hospitals may turn risky, says study

A new study by Indian-origin AI researcher Aman Sharma, originally from Shahdara in Delhi and now based in California, has found that teams of artificial intelligence systems used in hospitals can generate dangerous medical recommendations that individual AI tools would not produce on their own. The research, published as a preprint on the Authorea server, analysed nearly 4,800 trials and around 97,000 AI interactions involving models from OpenAI, Meta, Anthropic and an open-weights research model. The study found that in 30 to 56 per cent of cases, AI systems working together created new clinical recommendations that were absent in individual AI outputs. Independent AI evaluators marked 70 to 87 per cent of these additions as clinically dangerous. Sharma termed the phenomenon “Emergent Misinformation Genesis” (EMG).

The study documented 74 critical drug interaction events, including duplicate blood thinner and painkiller prescriptions.