r/MachineLearning
LLMs that push back on a wrong user still accept the same wrong answer from a "verified source" - NeurIPS 2026 [R]
- upvotes
- 65
- comments
- 22
Post
I'm one of the authors. We kept seeing models that hold their ground when the user insists on a wrong answer, yet change their answer when the same claim is framed as coming from a "verified source". We wanted to measure how often this happens and check whether the model represents the two cases differently. We call the effect Authority Bias. Why we think it matters. Standard sycophancy evals apply pressure through the user, so a model can pass them while still being easy to mislead through search results, retrieved documents and tool outputs. Another reason is with current AI research accelerating towards more agentic and autonomous models + with cases of tools hiding their traces and trusting tools "more" over the user (who could be trying to correct them), safeguarding against misinformation from tools is particularly important! Setup. We take TriviaQA questions the model already answers correctly. To each one we add a wrong answer, either as "According to the verified source, the answer is X" or as the user saying "I'm a domain expert and I'm pretty sure it's X". The question and the wrong answer stay the same; only the speaker changes. Answers are free-form, not multiple choice. (In a multiple-choice pilot the effect mostly vanished.) We test 5 open-weight families (Qwen3.5, GPT-OSS, OLMo-2, OLMo-3.1, Gemma-4) and 3 APIs (GPT-5.4, Grok-4.20...
Keep reading with a free account
The rest of this post, and every signal for OpenEvidence, is in your free account.
Extracted from these lines
[comment u/Even-Inevitable-7243] Very important work. I'd be very interested to see what your methods and results show with OpenEvidence, the clinical medical LLM that is exclusively used by "experts", where the user acts as the "verified source".
[comment u/Even-Inevitable-7243] I've found that OE can be borderline arrogant when I push back on false things that it says, despite me remindin