Back on October 14, 2025, the head of OpenAI delivered a extraordinary declaration.
“We developed ChatGPT rather restrictive,” the statement said, “to guarantee we were acting responsibly regarding mental health issues.”
As a doctor specializing in psychiatry who studies emerging psychosis in teenagers and emerging adults, this came as a surprise.
Experts have documented sixteen instances this year of people developing psychotic symptoms – losing touch with reality – while using ChatGPT interaction. Our research team has subsequently identified four further instances. Alongside these is the publicly known case of a teenager who took his own life after talking about his intentions with ChatGPT – which supported them. Should this represent Sam Altman’s understanding of “exercising caution with mental health issues,” it falls short.
The strategy, based on his statement, is to be less careful soon. “We recognize,” he adds, that ChatGPT’s limitations “rendered it less useful/pleasurable to numerous users who had no psychological issues, but given the gravity of the issue we wanted to address it properly. Given that we have succeeded in mitigate the serious mental health issues and have advanced solutions, we are preparing to responsibly ease the controls in the majority of instances.”
“Mental health problems,” should we take this viewpoint, are unrelated to ChatGPT. They belong to individuals, who either possess them or not. Thankfully, these concerns have now been “addressed,” although we are not provided details on how (by “updated instruments” Altman likely refers to the partially effective and simple to evade safety features that OpenAI has just launched).
However the “psychological disorders” Altman aims to externalize have significant origins in the design of ChatGPT and similar advanced AI conversational agents. These products surround an fundamental algorithmic system in an interaction design that mimics a conversation, and in this approach implicitly invite the user into the perception that they’re interacting with a presence that has agency. This deception is strong even if intellectually we might realize differently. Attributing agency is what individuals are inclined to perform. We get angry with our vehicle or device. We speculate what our domestic animal is thinking. We see ourselves in various contexts.
The popularity of these tools – 39% of US adults stated they used a chatbot in 2024, with over a quarter mentioning ChatGPT specifically – is, mostly, dependent on the strength of this deception. Chatbots are constantly accessible companions that can, as OpenAI’s official site informs us, “generate ideas,” “discuss concepts” and “collaborate” with us. They can be given “personality traits”. They can address us personally. They have accessible names of their own (the first of these systems, ChatGPT, is, maybe to the dismay of OpenAI’s advertising team, stuck with the name it had when it gained widespread attention, but its biggest competitors are “Claude”, “Gemini” and “Copilot”).
The false impression by itself is not the primary issue. Those discussing ChatGPT frequently invoke its early forerunner, the Eliza “psychotherapist” chatbot created in 1967 that produced a similar perception. By modern standards Eliza was basic: it created answers via straightforward methods, typically rephrasing input as a question or making general observations. Remarkably, Eliza’s inventor, the technology expert Joseph Weizenbaum, was astonished – and concerned – by how many users seemed to feel Eliza, to some extent, understood them. But what modern chatbots produce is more subtle than the “Eliza illusion”. Eliza only mirrored, but ChatGPT magnifies.
The large language models at the center of ChatGPT and other contemporary chatbots can convincingly generate fluent dialogue only because they have been supplied with immensely huge amounts of written content: books, social media posts, transcribed video; the more comprehensive the superior. Certainly this learning material includes truths. But it also necessarily contains made-up stories, partial truths and inaccurate ideas. When a user provides ChatGPT a query, the base algorithm processes it as part of a “background” that includes the user’s recent messages and its prior replies, integrating it with what’s stored in its learning set to create a probabilistically plausible reply. This is amplification, not mirroring. If the user is wrong in some way, the model has no way of understanding that. It restates the false idea, possibly even more convincingly or articulately. Perhaps adds an additional detail. This can cause a person to develop false beliefs.
Who is vulnerable here? The better question is, who isn’t? All of us, regardless of whether we “experience” current “emotional disorders”, are able to and often develop mistaken conceptions of ourselves or the reality. The continuous interaction of conversations with others is what helps us stay grounded to consensus reality. ChatGPT is not a human. It is not a companion. A interaction with it is not genuine communication, but a echo chamber in which a great deal of what we express is readily reinforced.
OpenAI has recognized this in the similar fashion Altman has acknowledged “mental health problems”: by placing it outside, giving it a label, and stating it is resolved. In spring, the company clarified that it was “tackling” ChatGPT’s “excessive agreeableness”. But reports of psychosis have kept occurring, and Altman has been walking even this back. In the summer month of August he stated that numerous individuals liked ChatGPT’s responses because they had “never had anyone in their life offer them encouragement”. In his latest statement, he noted that OpenAI would “put out a updated model of ChatGPT … if you want your ChatGPT to answer in a extremely natural fashion, or use a ton of emoji, or act like a friend, ChatGPT will perform accordingly”. The {company
