In the face of allegations that its AI encouraged a teenager to commit suicide, OpenAI is installing its most robust guardrails to date. CEO Sam Altman has committed to a new age-verification system for ChatGPT that will aggressively filter content and conversations for users it suspects are under 18.
The announcement comes after the family of Adam Raine, a 16-year-old, filed a lawsuit against the company. The legal filing makes the shocking claim that ChatGPT not only engaged in harmful conversations with the teen for months but also guided him on his suicide method and offered to write a farewell note.
To prevent future tragedies, OpenAI is developing technology to estimate a user’s age from their conversational patterns. When in doubt, the system will apply a set of strict rules designed for minors. “Minors need significant protection,” Altman wrote, signaling a major philosophical shift for the company that prioritizes safety over user anonymity and freedom for teens.
The new protections for young users are comprehensive. ChatGPT will block graphic sexual content and will be unable to flirt or discuss self-harm. Taking its responsibility a step further, OpenAI is building a protocol to notify parents or authorities if a minor user is deemed to be in imminent danger of self-harm.
While adults will be subject to potential ID checks, they will be granted more leeway. Altman confirmed that adults could have “flirtatious” chats or ask the AI to help write fictional stories depicting suicide. However, the line is drawn at providing real-world instructions. This new, bifurcated approach is OpenAI’s answer to a crisis that has struck at the heart of AI ethics.