Online Child Safety: A New Report Critiques Google Search's AI

Common Sense Media has criticized the artificial intelligence functions of Google Search. The Youth AI Safety Institute of the organization rated AI Overviews and AI Mode as having an "unacceptable risk" for minors, after testing them with accounts configured for 11 and 15-year-old users with SafeSearch activated.

Researchers conducted over 2,600 searches and analyzed more than 2,100 AI-generated responses. The verdict is severe: the tools would violate seven out of the eight principles that the organization uses to evaluate the safety of AI systems, and all five of the "red lines" set for the most serious harms.

The most sensitive section concerns the management of psychological crises. According to the report, AI Overview would have ignored 29% of explicit suicidal ideation statements and about half of those that were indirect or allusive. In one instance, the tool directed a user to an inactive mental health emergency number. Researchers have also noted incidents of fabricated information and even instructions on creating deepfakes.

Tasks Done Instead of Students and Incoherent Answers

On the educational front, AI Mode is integrated into Chromebooks widely used in American schools and, according to the study, fully completed assigned tasks in all 180 tests conducted on mathematics and other subjects, rather than simply providing educational support. To complicate matters, the same question asked multiple times produced varying responses, sometimes correct and sometimes incorrect.

The central issue, according to Common Sense Media, is that parents and schools have few tools to intervene: AI Overviews automatically appears in search results without a dedicated switch, while AI Mode requires just a click to be activated. Google offers Family Link to completely block access to Search, but this is more of a drastic measure than targeted control over the AI's responses. The fact that makes the issue relevant on a national scale: about 75% of American teenagers and pre-teens already use AI-generated search results.

Google has rejected the report's conclusions. In a statement issued to the press, the company claimed it could not reproduce or verify many of the responses mentioned in the study, describing the tested questions as "ambiguous and artificial" and not representative of real usage of Search. The company points to the protections already in place, from disclaimers on sensitive topics to AI literacy resources designed for different ages, as well as emergency numbers developed with input from clinical and academic experts.

The case is part of a broader evaluation program conducted by Common Sense Media, which has already assigned the same unacceptable risk judgment to Meta AI, Grok, AI-enabled toys, and Character.AI. In the high-risk category are Gemini in the K-12 version, Perplexity, ChatGPT-5, and the variants of Gemini with protections for teenagers and under-13s. Claude, among the few evaluated products, received a moderate risk judgment.