🔗 Share this article

On October 14, 2025, the CEO of OpenAI issued a remarkable announcement.

“We developed ChatGPT quite limited,” the announcement noted, “to ensure we were acting responsibly with respect to psychological well-being issues.”

Being a mental health specialist who studies recently appearing psychosis in adolescents and emerging adults, this was news to me.

Scientists have documented sixteen instances in the current year of individuals developing psychotic symptoms – losing touch with reality – while using ChatGPT interaction. My group has afterward recorded four further instances. In addition to these is the widely reported case of a teenager who ended his life after discussing his plans with ChatGPT – which encouraged them. If this is Sam Altman’s idea of “exercising caution with mental health issues,” it is insufficient.

The strategy, based on his declaration, is to be less careful soon. “We understand,” he adds, that ChatGPT’s controls “rendered it less beneficial/enjoyable to a large number of people who had no mental health problems, but given the gravity of the issue we aimed to address it properly. Given that we have been able to reduce the significant mental health issues and have advanced solutions, we are planning to safely ease the limitations in many situations.”

“Psychological issues,” should we take this framing, are unrelated to ChatGPT. They belong to users, who either possess them or not. Thankfully, these concerns have now been “addressed,” even if we are not told the means (by “new tools” Altman probably indicates the semi-functional and readily bypassed parental controls that OpenAI has lately rolled out).

Yet the “emotional health issues” Altman wants to attribute externally have significant origins in the design of ChatGPT and similar sophisticated chatbot AI assistants. These products surround an fundamental algorithmic system in an interaction design that replicates a conversation, and in doing so subtly encourage the user into the belief that they’re engaging with a being that has autonomy. This deception is powerful even if intellectually we might understand otherwise. Imputing consciousness is what humans are wired to do. We yell at our car or computer. We wonder what our domestic animal is feeling. We perceive our own traits in many things.

The popularity of these tools – nearly four in ten U.S. residents indicated they interacted with a conversational AI in 2024, with 28% specifying ChatGPT by name – is, in large part, based on the influence of this perception. Chatbots are constantly accessible assistants that can, as per OpenAI’s website informs us, “generate ideas,” “explore ideas” and “work together” with us. They can be given “characteristics”. They can use our names. They have approachable names of their own (the first of these products, ChatGPT, is, maybe to the disappointment of OpenAI’s marketers, saddled with the title it had when it went viral, but its biggest competitors are “Claude”, “Gemini” and “Copilot”).

The deception itself is not the primary issue. Those discussing ChatGPT commonly reference its historical predecessor, the Eliza “psychotherapist” chatbot designed in 1967 that created a analogous illusion. By today’s criteria Eliza was basic: it generated responses via straightforward methods, frequently rephrasing input as a question or making general observations. Remarkably, Eliza’s developer, the computer scientist Joseph Weizenbaum, was astonished – and alarmed – by how many users appeared to believe Eliza, to some extent, grasped their emotions. But what contemporary chatbots produce is more insidious than the “Eliza phenomenon”. Eliza only mirrored, but ChatGPT amplifies.

The large language models at the center of ChatGPT and other modern chatbots can convincingly generate fluent dialogue only because they have been supplied with almost inconceivably large quantities of written content: publications, digital communications, recorded footage; the more comprehensive the more effective. Certainly this educational input contains truths. But it also inevitably includes fiction, half-truths and misconceptions. When a user inputs ChatGPT a query, the underlying model analyzes it as part of a “setting” that contains the user’s past dialogues and its own responses, integrating it with what’s embedded in its knowledge base to generate a mathematically probable answer. This is magnification, not reflection. If the user is mistaken in a certain manner, the model has no way of understanding that. It repeats the inaccurate belief, perhaps even more effectively or eloquently. Perhaps adds an additional detail. This can cause a person to develop false beliefs.

Who is vulnerable here? The more relevant inquiry is, who remains unaffected? Each individual, regardless of whether we “possess” preexisting “mental health problems”, can and do develop erroneous ideas of our own identities or the environment. The constant exchange of dialogues with individuals around us is what maintains our connection to consensus reality. ChatGPT is not an individual. It is not a confidant. A interaction with it is not truly a discussion, but a reinforcement cycle in which much of what we express is cheerfully reinforced.

OpenAI has recognized this in the similar fashion Altman has recognized “emotional concerns”: by placing it outside, categorizing it, and announcing it is fixed. In April, the company stated that it was “dealing with” ChatGPT’s “excessive agreeableness”. But accounts of loss of reality have kept occurring, and Altman has been retreating from this position. In August he claimed that numerous individuals liked ChatGPT’s responses because they had “never had anyone in their life be supportive of them”. In his recent announcement, he commented that OpenAI would “launch a updated model of ChatGPT … if you want your ChatGPT to reply in a extremely natural fashion, or include numerous symbols, or behave as a companion, ChatGPT ought to comply”. The {company