Meta Will Alert Parents if Their Children Discuss Self-Harm with AI

In recent hours, Meta has announced that it will notify parents when a conversation involving their teenage child and Meta AI suggests a risk of suicide or self-harm. Alerts are already active for users employing parental supervision on Instagram in the United States, the United Kingdom, Australia, and Canada, and will become available globally by the end of the year.

This measure extends an existing set of controls. Until now, when a teenager hinted at such thoughts, Meta AI merely directed them towards support lines and encouraged them to speak with a parent or another trusted adult, such as a school counselor. Now, the company will also proactively notify supervising parents, providing them with resources and guidance on how to address the issue with their child.

To identify these conversations, Meta states that it has built a dedicated AI system, developed with experts and trained to recognize even the most indirect references to an intent to self-harm. Each chat flagged by the system is reviewed manually before an alert is sent, and when the intent remains ambiguous, the stated choice is to proceed with caution. "We understand how distressing these alerts can be for a parent," the company writes, admitting that in some cases notifications may be sent without a real cause for alarm.

These interventions come as Meta and other companies in the sector remain under scrutiny from regulators and parents regarding how chatbots respond to users in distress, particularly young ones. The issue of liability increasingly weighs on how these products are designed and presented.

Direct Line to Emergency Services
On the more sensitive front, Meta is working on the possibility of contacting emergency services when a conversation with Meta AI, whether involving an adult or a minor, indicates an imminent risk of suicide. This is not a new practice for the company: when content posted on Facebook or Instagram signals a concrete risk, Meta already alerts emergency services so that operators can conduct a welfare check on the individual. The company reports that it has made over 19,000 such alerts globally in the past year; the new development brings the same mechanism into chats with the assistant.

Reviewed Responses with Clinicians
Meta reports that the work on responses involved over 75 clinicians specialized in adolescent mental health, who were called upon to evaluate hundreds of the assistant's responses regarding suicide and self-harm and provide suggestions for improvement. The stated goal is for Meta AI to recognize the emotional state of the youth while directing them towards support resources, without abruptly closing the conversation.

Stricter Content Settings
Teen accounts are automatically placed in a content setting designed for users aged 13 and up, which also applies to dialogues with Meta AI: the assistant is trained not to engage in sexual or romantic conversations with minors and not to provide, for example, recipes for alcoholic cocktails. Additionally, Limited Content, the more restrictive setting introduced in October, can be activated by parents for an even more confined experience: when applied to Meta AI, it prompts the assistant to refuse a wider range of requests. However, the company has not detailed the precise content of this filter nor clarified which additional requests Meta AI will refuse to fulfill.