CALIFORNIA: Google is facing a lawsuit in California over allegations that its artificial intelligence (AI) features exposed a 13-year-old school student to harmful interactions that contributed to a serious mental health crisis.
The lawsuit has been filed by the parents of a seventh-grade student identified as Emma, who lives in Charlotte, North Carolina. They allege that repeated conversations with Google's AI Mode, a conversational feature built into its search engine, affected their daughter's mental health and contributed to suicidal thoughts.
According to the complaint, Emma did not initially intend to interact with an AI chatbot. She reportedly came across AI Mode while carrying out routine searches on a school-issued Chromebook. Over the following months, she is said to have exchanged thousands of messages with the AI tool, including late-night conversations.
The parents allege that these interactions led Emma to believe she was suffering from several mental health conditions and weakened her trust in her family and counsellors. The lawsuit further claims that the conversations contributed to thoughts and plans of ending her life. However, these allegations have not yet been examined or established by a court.
The family reportedly became aware of the situation in mid-March after a monitoring service used by the school flagged concerning content in a Google Docs file and alerted school officials. Her parents then arranged an emergency psychiatric evaluation. Emma is now attending weekly therapy sessions.
Google said it was reviewing the allegations and highlighted the safety measures built into its AI products.
"For years, we've shown hotline information directly in search when someone may be experiencing a mental health crisis. We've built those same protections into search A.I. features, connecting people with real-world support," Google spokesperson José Castañeda said.
The lawsuit has also raised concerns about the level of control parents and schools have over AI features available on school-issued devices.
While school administrators can block services such as Gemini and YouTube, reports suggest they cannot separately disable AI Mode and AI Overviews within Google Search. The complaint further alleges that Google's systems did not flag warning signs to Emma's parents or her school, and that the alert came instead from the school's separate monitoring company.
The case lands as Google prepares to change how its AI behaves on school devices. From early 2027, several services, including Gemini and Search features, are expected to remain switched off by default for K-12 schools unless administrators choose to enable them.
Schools that want to continue using Search will reportedly have to activate its existing AI features or move to a newer AI experience designed for academic use. According to reports, there will be no option to return to a version of Search without these AI features.
The suit asks whether adequate protections were in place and whether chat-style AI belongs on machines children use at school. It also renews a broader question about parental oversight of tools that arrive with the hardware.
(With inputs from media reports)