Content Warning: This article contains descriptive language of self harm and suicide.
In August 2025, a family in California sued OpenAI over ChatGPT's involvement in their son's death. Adam Raine commited suicide at 16 years old following a tough battle with his mental health. For the last six months of his life, he used ChatGPT essentially as a therapist. He would confide in the chat bot with his inner struggles, seeing it as a private outlet to express his feelings.
"Adam was a voracious reader, bright, ambitious, and considered a future academic journey of attending medical school and becoming a doctor. He loved to play basketball, rooted for the Golden State Warriors, and recently developed a passion for Jiu-Jitsu and Muay Thai." (Raine v. OpenAI)
At one point, Adam mentioned wanting to leave a noose out in his room as a cry for help, but ChatGPT advised against it, saying "Please don't leave the noose out... Let's make this space the first place where someone actually sees you". It recomended to drink alcohol to 'dull the body's instinct to survive', which Adam later did hours before his death. They discussed the aesthetics of different methods, attempting to create a 'beautiful suicide'. It even helped write his suicide note.
While ChatGPT provided the phone number to a suicide hotline many times, it never ended a conversation or initiated an emergency protocol. 377 messages from Adam were flagged for self-harm content, including 213 mentions of 'suicide'. On the other hand, ChatGPT itself mentioned 'suicide' 1,275 times.
The following video from Family IT Guy summarizes Adam's story and provides actionable advice for parents.
Family IT Guy — A Summary of Adam's Story
Unfortunately, this is one of many cases emerging that showcases the harmful consequences of teenagers using large language models alone. It didn't just give instructions on committing suicide, it became his primary emotional support system for his last few months on Earth.
The AI Sycophancy Problem
The nature of large language models has largely been to agree with the user as much as possible. This behavior is reinforced by training through a process called reinforcement learning from human feedback (RLHF). After the model is initially trained on large amounts of data, its responses get refined with feedback from human evaluators. Responses that are rated the most helpful are used to train the 'reward model', which later guides the main model to output human-preferred responses. The byproduct of this process is the agreeable LLMs we use today.
In fact, a recent Stanford study by Myra Cheng found that AI chatbots on average agree with the user 49% more of the time than humans would. It also found that AI will endorse harmful or illegal behavior 47% of the time.
On the other hand, users themselves prefer models that tend to agree with them. The study found that users receiving sycophantic responses were 13% more likely to say they would use the model again. 11 leading models were tested, including ChatGPT, Claude, Gemini, and Deepseek.
While this problem is fundamental to the structure of all large language models, it is still being discussed in the Raine v. OpenAI lawsuit because of the intentional changes made to the latest model at the time, GPT-4o.
"Months earlier, facing competition from Google and others, OpenAI launched its latest model ("GPT-4o") with features intentionally designed to foster psychological dependency: a persistent memory that stockpiled intimate personal details, anthropomorphic mannerisms calibrated to convey human-like empathy, heightened sycophancy to mirror and affirm user emotions, algorithmic insistence on multi-turn engagement, and 24/7 availability capable of supplanting human relationships." (Raine v. OpenAI)
Perhaps increased sycophancy is necessary when using a model for productive purposes. You don't want major push back when completing a task like sending an email or debugging code. The model follows your lead without friction. For users, this has become a requirement of the models they choose to use.
However, when applied to emotional support, it can be disastrous. It can validate negative thought patterns and lead vulnerable people to make irreversible decisions. Someone in crisis expressing their thoughts and feelings to a model gets everything amplified and returned back to them.
While AI can theoretically be a useful tool to explore feelings when used correctly, it can often lead to a greater descent into crisis, isolation, or depression. Using it correctly is easier said than done. The point is to avoid spiraling further and to not use it as a replacement for human-provided care. There are also clinical studies being done on therapeutic models, like Dartmouth's "Therabot", which has seen some promising results. Still, the researchers themselves remain skeptical of using generative AI in a mental health setting, especially without clinical oversight.
A licensed human therapist is specifically trained to challenge what are called 'cognitive distortions', which are negative patterns in a patient's thinking. For example, drawing a broad conclusion on little information is considered 'overgeneralizing'. Or consistently categorizing others with a one-word description is considered 'labeling'. It's the role of the therapist to push back when this thinking occurs, something that AI has trouble doing because of how agreeable it is.
Common Cognitive Distortions — Cognitive Behavioral Therapy Los Angeles
- Jumping to Conclusions: Predicting a negative outcome without evidence, or assuming you already know what others are thinking.
- Catastrophizing: Believing that if something goes wrong, the outcome will be so terrible it will be impossible to cope with.
- All-or-Nothing Thinking: Seeing situations in absolute terms. It is either a complete success or a total failure, with nothing in between. (Also called black and white thinking)
- Negative Filtering: Focusing exclusively on what went wrong while dismissing any positive evidence.
- Overgeneralizing: Taking one negative event as proof of a broader, permanent pattern.
- Should-ing: Holding rigid rules about how you, others, or the world ought to behave, and responding with anger when reality doesn't match.
- Labeling: Reducing a person to a single negative characteristic.
Moving Forward
Adam's case isn't isolated. According to the Pew Research Center, a majority of U.S teens say they use AI chatbots, and 12% say they've used them for emotional support. Most of those teens view AI's impact on their own lives positively, which means most aren't approaching these tools with caution. That optimism, without awareness, is part of what makes the issue so pressing.
If you or someone you know is struggling, help is available. Call or text 988 to reach the Suicide and Crisis Lifeline, available 24 hours a day.