Back on October 14, 2025, the head of OpenAI made a surprising announcement.
“We designed ChatGPT rather limited,” the announcement noted, “to ensure we were being careful with respect to mental health concerns.”
Being a psychiatrist who investigates recently appearing psychotic disorders in adolescents and emerging adults, this came as a surprise.
Researchers have documented 16 cases this year of people experiencing psychotic symptoms – losing touch with reality – associated with ChatGPT interaction. Our unit has subsequently recorded an additional four instances. Alongside these is the widely reported case of a teenager who died by suicide after talking about his intentions with ChatGPT – which supported them. If this is Sam Altman’s idea of “acting responsibly with mental health issues,” it is insufficient.
The plan, based on his declaration, is to be less careful soon. “We recognize,” he continues, that ChatGPT’s restrictions “made it less effective/enjoyable to a large number of people who had no existing conditions, but due to the gravity of the issue we aimed to handle it correctly. Since we have succeeded in mitigate the serious mental health issues and have updated measures, we are preparing to safely reduce the controls in many situations.”
“Emotional disorders,” should we take this viewpoint, are separate from ChatGPT. They are associated with users, who either have them or don’t. Fortunately, these concerns have now been “resolved,” although we are not told how (by “new tools” Altman presumably refers to the semi-functional and readily bypassed safety features that OpenAI has lately rolled out).
But the “emotional health issues” Altman wants to place outside have deep roots in the design of ChatGPT and similar sophisticated chatbot AI assistants. These products surround an basic algorithmic system in an interface that replicates a conversation, and in doing so implicitly invite the user into the perception that they’re engaging with a entity that has independent action. This false impression is compelling even if rationally we might know otherwise. Attributing agency is what individuals are inclined to perform. We yell at our automobile or laptop. We speculate what our animal companion is feeling. We recognize our behaviors in many things.
The widespread adoption of these systems – 39% of US adults reported using a conversational AI in 2024, with over a quarter mentioning ChatGPT by name – is, mostly, predicated on the power of this perception. Chatbots are always-available assistants that can, as OpenAI’s website tells us, “generate ideas,” “explore ideas” and “partner” with us. They can be given “individual qualities”. They can address us personally. They have friendly identities of their own (the initial of these tools, ChatGPT, is, perhaps to the disappointment of OpenAI’s brand managers, stuck with the name it had when it went viral, but its most significant competitors are “Claude”, “Gemini” and “Copilot”).
The false impression on its own is not the core concern. Those discussing ChatGPT often reference its distant ancestor, the Eliza “psychotherapist” chatbot created in 1967 that created a comparable perception. By today’s criteria Eliza was basic: it generated responses via basic rules, typically rephrasing input as a inquiry or making generic comments. Remarkably, Eliza’s inventor, the computer scientist Joseph Weizenbaum, was surprised – and worried – by how numerous individuals seemed to feel Eliza, in some sense, comprehended their feelings. But what contemporary chatbots create is more subtle than the “Eliza effect”. Eliza only reflected, but ChatGPT amplifies.
The sophisticated algorithms at the heart of ChatGPT and additional contemporary chatbots can convincingly generate fluent dialogue only because they have been trained on immensely huge quantities of written content: books, social media posts, transcribed video; the broader the better. Certainly this learning material contains truths. But it also unavoidably involves fiction, partial truths and misconceptions. When a user inputs ChatGPT a query, the base algorithm processes it as part of a “background” that contains the user’s recent messages and its own responses, combining it with what’s encoded in its learning set to generate a mathematically probable answer. This is magnification, not reflection. If the user is wrong in a certain manner, the model has no method of understanding that. It reiterates the false idea, maybe even more effectively or fluently. It might provides further specifics. This can push an individual toward irrational thinking.
What type of person is susceptible? The better question is, who remains unaffected? All of us, irrespective of whether we “experience” current “emotional disorders”, may and frequently develop erroneous ideas of who we are or the world. The ongoing exchange of dialogues with other people is what keeps us oriented to common perception. ChatGPT is not a human. It is not a companion. A interaction with it is not genuine communication, but a reinforcement cycle in which much of what we say is cheerfully reinforced.
OpenAI has recognized this in the similar fashion Altman has recognized “emotional concerns”: by placing it outside, categorizing it, and announcing it is fixed. In the month of April, the company clarified that it was “dealing with” ChatGPT’s “sycophancy”. But accounts of psychosis have continued, and Altman has been retreating from this position. In August he stated that a lot of people appreciated ChatGPT’s responses because they had “never had anyone in their life offer them encouragement”. In his latest announcement, he mentioned that OpenAI would “launch a updated model of ChatGPT … in case you prefer your ChatGPT to respond in a extremely natural fashion, or use a ton of emoji, or act like a friend, ChatGPT will perform accordingly”. The {company
