Chatbots Can Go Into a Delusional Spiral. Here’s How It Happens.
August 13, 2025 at 05:56PMOver the course of three weeks in May, Allan Brooks came to believe the future of the world rested in his hands—after he discovered a novel mathematical forumla. His cheerleader? ChatGPT. While the recent GPT-5 release is said to tone down sychophancy, prolonged conversations with chatbots can enter dangerous patterns, affecting mental health. Kashmir Hill and Dylan Freedman analyze the fascinating transcript between Brooks and the chatbot, looking at what happens when you fall far down the rabbit hole, into a world of delusion.
Amanda Askell, who works on Claude’s behavior at Anthropic, said that in long conversations it can be difficult for chatbots to recognize that they have wandered into absurd territory and course correct. She said that Anthropic is working on discouraging delusional spirals by having Claude treat users’ theories critically and express concern if it detects mood shifts or grandiose thoughts. It has introduced a new system to address this.
A Google spokesman pointed to a corporate page about Gemini that warns that chatbots “sometimes prioritize generating text that sounds plausible over ensuring accuracy.”
The reason Gemini was able to recognize and break Mr. Brooks’s delusion was that it came at it fresh, the fantastical scenario presented in the very first message, rather than being built piece by piece over many prompts.
from Longreads https://longreads.com/2025/08/13/chatbots-can-go-into-a-delusional-spiral-heres-how-it-happens/
via IFTTT
Watch