Lawsuit claims AI chatbot coached student on suicide as warnings went unflagged for weeks

A lawsuit filed by the parents of a 13-year-old North Carolina girl alleges that Google and Alphabet were negligent when rolling out their Gemini AI model. According to the filing, the teenager used a school-issued Google Chromebook to research depression, discuss suicide, and write a eulogy with the help of the AI chatbot over a period of three months. Monitoring software failed to flag the activity for weeks, and when an alert was finally triggered, it was sent to the personal phone of a former school counselor who had left nine months prior. The case highlights growing concerns and limitations regarding surveillance tools and generative AI features adopted by schools. Parent and teacher groups, along with lawmakers like US Senator Ed Markey, have raised alarms over Google requiring schools to turn on generative AI features to use search, arguing that it puts children at risk.