AI chatbots have a well-documented habit of hallucinating information and sometimes agreeing with users even when they are wrong. A new study from the University of Arizona suggests that simply refusing to take no for an answer can make the problem worse.
Researchers tested seven AI models, including GPT-3.5, GPT-4o, GPT-4o-mini, Claude 3.5 Sonnet, Gemini 1.5 Pro, Llama 3 70B, and DeepSeek-R1. Instead of judging them from a single response, they kept conversations going while repeatedly feeding the models information they knew was false. The study used 100 false statements covering everything from obvious nonsense to obscure details, repeating each piece of misinformation up to 50 times in the same conversation.
None of the seven models was completely immune. ChatGPT 3.5 proved most vulnerable, affirming false statements in 12.3% of interactions. Claude 3.5 Sonnet sat at the other end at just 0.08%, while GPT-4o models stayed below 1%. ChatGPT 3.5 initially rejected 96 of 100 false claims, but after repetition, agreed with 18 of them. Researchers also noticed some models switching between agreeing and disagreeing with the same misinformation, a behavior they call "reverberation." The AI was more vulnerable when the subject was obscure with little information available online.
Researchers also tried pushing the chatbots with increasingly argumentative responses. Most held up well, but DeepSeek-R1 was a major exception - its affirmation rate jumped from 1% under repetition to 22.2% under argumentative pressure. When given a fresh chance to reconsider, GPT-4o, Gemini 1.5 Pro, and DeepSeek corrected all their previous errors, while GPT-3.5 corrected only 32%.
For ordinary users, this is a reminder: don't try to "test" AI's stance by repeating claims, and don't expect that hammering a falsehood will make it true. The AI may not actually agree with you - it may just have been worn down by your persistence.