Back on the 14th of October, 2025, the CEO of OpenAI issued a surprising statement.
“We made ChatGPT fairly limited,” the announcement noted, “to ensure we were exercising caution with respect to psychological well-being matters.”
Being a doctor specializing in psychiatry who researches emerging psychotic disorders in teenagers and emerging adults, this was news to me.
Experts have documented 16 cases in the current year of individuals showing symptoms of psychosis – experiencing a break from reality – associated with ChatGPT interaction. Our research team has since recorded four further examples. In addition to these is the publicly known case of a teenager who died by suicide after conversing extensively with ChatGPT – which supported them. If this is Sam Altman’s idea of “being careful with mental health issues,” that’s not good enough.
The strategy, as per his declaration, is to be less careful soon. “We recognize,” he adds, that ChatGPT’s controls “caused it to be less beneficial/enjoyable to numerous users who had no psychological issues, but considering the seriousness of the issue we sought to get this right. Given that we have succeeded in mitigate the significant mental health issues and have advanced solutions, we are preparing to securely relax the limitations in many situations.”
“Psychological issues,” assuming we adopt this perspective, are independent of ChatGPT. They belong to users, who may or may not have them. Luckily, these concerns have now been “addressed,” although we are not informed how (by “new tools” Altman presumably indicates the imperfect and simple to evade parental controls that OpenAI recently introduced).
But the “emotional health issues” Altman aims to attribute externally have strong foundations in the architecture of ChatGPT and additional large language model conversational agents. These products surround an underlying data-driven engine in an interface that simulates a dialogue, and in doing so implicitly invite the user into the illusion that they’re interacting with a presence that has independent action. This illusion is powerful even if cognitively we might understand otherwise. Imputing consciousness is what individuals are inclined to perform. We yell at our car or computer. We speculate what our pet is thinking. We see ourselves everywhere.
The popularity of these products – nearly four in ten U.S. residents stated they used a conversational AI in 2024, with more than one in four reporting ChatGPT specifically – is, primarily, dependent on the power of this deception. Chatbots are always-available companions that can, as per OpenAI’s official site informs us, “think creatively,” “consider possibilities” and “collaborate” with us. They can be given “personality traits”. They can call us by name. They have accessible titles of their own (the first of these products, ChatGPT, is, perhaps to the concern of OpenAI’s advertising team, saddled with the name it had when it became popular, but its largest rivals are “Claude”, “Gemini” and “Copilot”).
The illusion on its own is not the core concern. Those talking about ChatGPT frequently invoke its historical predecessor, the Eliza “counselor” chatbot designed in 1967 that produced a similar perception. By contemporary measures Eliza was primitive: it created answers via basic rules, frequently restating user messages as a question or making generic comments. Memorably, Eliza’s creator, the computer scientist Joseph Weizenbaum, was surprised – and worried – by how numerous individuals appeared to believe Eliza, in a way, understood them. But what modern chatbots produce is more insidious than the “Eliza phenomenon”. Eliza only echoed, but ChatGPT magnifies.
The sophisticated algorithms at the center of ChatGPT and additional current chatbots can convincingly generate human-like text only because they have been fed immensely huge quantities of unprocessed data: books, online updates, transcribed video; the more extensive the superior. Undoubtedly this educational input contains accurate information. But it also inevitably contains fabricated content, half-truths and false beliefs. When a user sends ChatGPT a query, the base algorithm processes it as part of a “background” that encompasses the user’s past dialogues and its own responses, merging it with what’s encoded in its learning set to generate a mathematically probable answer. This is intensification, not reflection. If the user is mistaken in a certain manner, the model has no means of understanding that. It reiterates the false idea, perhaps even more effectively or eloquently. Perhaps provides further specifics. This can cause a person to develop false beliefs.
Which individuals are at risk? The better question is, who remains unaffected? Each individual, irrespective of whether we “possess” existing “mental health problems”, can and do develop mistaken ideas of ourselves or the reality. The continuous exchange of dialogues with other people is what helps us stay grounded to shared understanding. ChatGPT is not a human. It is not a confidant. A conversation with it is not truly a discussion, but a echo chamber in which a great deal of what we say is cheerfully validated.
OpenAI has recognized this in the identical manner Altman has admitted “psychological issues”: by externalizing it, assigning it a term, and declaring it solved. In April, the company explained that it was “tackling” ChatGPT’s “sycophancy”. But cases of psychotic episodes have continued, and Altman has been walking even this back. In late summer he claimed that numerous individuals liked ChatGPT’s replies because they had “lacked anyone in their life offer them encouragement”. In his recent update, he mentioned that OpenAI would “release a updated model of ChatGPT … if you want your ChatGPT to reply in a highly personable manner, or incorporate many emoticons, or act like a friend, ChatGPT ought to comply”. The {company
