CALIFRONIA: A legal action filed in California Superior Court on Novermber 4 accuses Google of negligence, claiming that artificial intelligence features integrated into its search engine played a major role in a 13-year-old student’s self-harm crisis on a school-provided laptop.
The lawsuit is considered the first known case challenging the safety of Google's search-based AI tools deployed on educational Chromebooks.
According to court filings, the student, identified as Emma from North Carolina, had been barred by school network filters from accessing standalone conversational platforms like OpenAI's ChatGPT and Google's Gemini. However, the regular Google Search interface continued to provide AI-generated responses and summaries, which were not part of the standard restrictions recently applied to student profiles.
Andrew Liddell, an attorney representing the family, stated to AFP that school districts should suspend search access altogether until service providers remove generative AI modes from search or implement sufficient safeguards for children interacting with automated systems.
The complaint describes the teenager as an engaged student with strong academic performance, musical interests, and involvement in the Girl Scouts, noting she had no prior history of mental health difficulties. The lawsuit contends that over a period of several months, using AI-assisted search tools on her school computer contributed to an unaddressed emotional crisis.
The situation was discovered on March 17 when automated school safety software flagged a document titled "Funeral" and alerted the family. The student's mother subsequently observed her daughter composing a farewell message on her account, enabling her father to intervene immediately and prevent self-harm.
(With inputs from media reports)