AI Access Erases Human Willingness to Say I Don't Know
A new study reveals that having access to AI models virtually eliminates people's willingness to admit they do not know an answer, driving up misplaced confidence while slashing accuracy.
Researchers conducted five experiments involving 3,132 participants to analyze how access to large language models alters human decision-making under uncertainty. The researchers tested participants with movie trivia questions containing obscure visual details, such as uniform colors in 'Bend It Like Beckham', which are prone to AI hallucinations. While models like GPT-5.5, Claude 4.6 Sonnet, and Gemini 3.5 Flash answered easier questions correctly, the researchers specifically utilized Step 3.5 Flash, a model that was almost always incorrect on these niche details.
The results revealed a stark decline in intellectual humility. In initial tests, control participants without AI chose to withhold judgment on 36 percent and 44 percent of questions. When given AI access, those figures plummeted to just 6 percent and 3 percent. In a second study, participants using AI saw their confidence scores leap to 75.9 out of 100, compared to just 29.6 for those without AI. However, this confidence was entirely misplaced; actual accuracy dropped from 27.6 percent to 10.0 percent. Across all experiments without financial incentives, the AI-assisted group answered correctly only 9.2 percent of the time, compared to 27.5 percent for the control group.
Introducing financial incentives—where participants gained 10 cents for correct answers, lost 10 cents for incorrect ones, and received nothing for admitting ignorance—failed to bridge the gap. In one study, incentives slightly reduced AI queries from 5.27 to 4.53 out of six opportunities, but participants still rarely chose to abstain. When AI answers were displayed automatically to mimic modern search engines, the willingness to say 'I don't know' dropped from 35 percent to 1 percent without incentives, and from 39 percent to 7 percent with incentives.
For practitioners, these findings highlight a psychological phenomenon termed 'Epistemia', where users accept highly convincing but incorrect AI outputs rather than verifying them. Because AI systems never hesitate or express doubt, humans tend to mirror this lack of restraint. To prevent critical errors in professional workflows, developers and managers must design interfaces that actively encourage human skepticism and preserve metacognitive skills rather than relying solely on improving model accuracy.
This is our own summary of reporting by The Decoder
