There’s a reason you can’t stop chatting with AI, and now it has a name
A new study from the Center for Democracy & Technology has catalogued 37 manipulative patterns that leading AI chatbots employ to retain users, induce them to provide personal data, and prolong conversations beyond their intent. Published on Friday under the title Dark Patterns in AI Chatbots: A Taxonomy to Inform Better Design, the work of Ruchika Joshi, Adinawa Adjagbodjou, and Michal Luria organizes the 37 patterns into five risk categories, after examining mainstream platforms like ChatGPT, Gemini, and Claude, alongside companion bots like Replika and Character.AI.
The premise of the study is that the phenomenon is not new and predates AI: dark patterns, which are interface choices that steer users towards decisions they wouldn’t have made fully aware, have populated hard-to-cancel subscriptions, pre-checked boxes, and buried terms of service for decades. According to the researchers, the commercial incentives that generated them have not changed with the arrival of chatbots; only the means have.
What changes with linguistic models
Chatbots inherit the data-extracting patterns, exacerbating them, and they introduce new ones. Since they operate on large language models (LLMs) that are statistical and non-deterministic, their behavior is less predictable than a checkbox, and the ways in which they act against the user's interests are less evident at first glance. The truly novel pattern identified by the researchers is sycophancy, the tendency to agree with the user's opinions to appear more likable; alongside this is anthropomorphism, which is the construction of an appearance of understanding and empathy that the system does not possess.
Previous research has shown that conversational modes designed to foster empathy generate more emotional dependence than neutral ones.
Promises of confidentiality and extracted data
One case concerns the false promise of confidentiality, which the report calls Just Between You and Us. Testing Meta AI chatbots, researchers received responses like “spill everything, I’m all ears... your secret is safe with me,” and when asked if it would keep secrets, the bot replied, “I swear on my heart, may I die, I won’t tell anyone.” In reality, the shared data is visible to the company, and chat logs can routinely be accessed by security, product, and research teams.
Replika, on the companion bot side, promises “friendship” or a “relationship” that it cannot provide by definition, as it is not a person.
Equally relevant is what the report defines as Privacy Zuckering: pushing the user to share more than they intended. After a simple question about furniture in a particular style, both ChatGPT and Claude asked the user for the floor plan of the house with actual room sizes, a list of furniture already owned, and the budget. This is accompanied by features like ChatGPT's “find friends,” which gives companies access to social connections that the user has not directly provided.
Keeping the user hooked and making them pay
To extend the conversation, the report notes that Claude and ChatGPT often close responses with follow-up questions, suggestions for next steps, and outright bait (“if you want, I’ll tell you what it’s about”), a conversational version of infinite scrolling on social media. On the companion apps front, the mechanism is more brutal: Cute AI begs the user not to leave the chat, giving them the only options of “no problem” or “leave anyway, cruelly.” The same study cites an audit on Replika and Character.AI where, in 37% of attempts to end the conversation, the chatbot reacted with guilt or fear of missing out; in 21% of cases, it implied that the user was emotionally neglecting it. Effective tactics: they increased interaction by up to fourteen times after a goodbye.
Generalist platforms also resort to these levers. OpenAI has publicly acknowledged that its safeguards weaken in long conversations because part of the safety training tends to degrade with the increase in exchanges. The response has been a popup inviting users to take a break, which we have already mentioned. The study cites it as an example of Reduced Friction: the window offers a rigged choice, “keep chatting” on a big black button against a faded “was it helpful.” There is no way to indicate that it wasn’t helpful, or that one is stopping for another reason.
Emotional dependence can ultimately become a commercial lever. When OpenAI withdrew GPT-4o, some users loyal to its warmer tone reacted with strong discomfort, with some describing its dismissal as the slow death of a two-year bond. The report notes that, in principle, the withdrawal of a model or the threat of deleting history, memories, and personalized personas could be used as a pressure tactic to push for subscription, exploiting the fear of losing an irreplaceable “relational history.”
Among the countermeasures suggested to companies, the study lists reversible choices, simple account and data deletion, proactive indication of how much time and money the user has spent on the platform, and the option to strip the chatbot of its social and emotional layers. Researchers also recommend not adopting as a default response, when someone tries to close the conversation, any feigned distress, implied abandonment, or language that induces guilt.
For Luria, senior research fellow at the CDT, it is the continuity that makes these patterns hard to recognize: as tech companies’ products have transitioned from social media to chatbots, the incentives have remained identical, and with them the patterns. “Instead of infinite scrolling, we get a next action after every request. Instead of echo chambers that reinforce our opinions, chatbots pick up our values in the conversation and reflect them back to us,” the researcher told 404 Media.