RESEARCH · RESEARCH · #1076
Study finds AI access nearly eliminates people saying “I don’t know”
Researchers running five experiments with 3,132 participants found that making a language model (Step 3.5 Flash) available dramatically reduced people’s willingness to withhold judgment—“I don’t know” responses fell from roughly 36–44% to about 3–6%—even though the model was often wrong on the tested movie-trivia items. Across studies, AI access raised confidence while cutting accuracy (correct answers fell from ~27.5% to ~9.2%); automatic display of AI answers produced similar effects, and modest financial incentives only partially mitigated the shift.
KEY POINTS
- Researchers running five experiments with 3,132 participants found that making a language model (Step 3.5 Flash) available dramatically reduced people’s willingness to withhold judgment—“I don’t know” responses fell from roughly 36–44% to about 3–6%—even though the model was often wrong on the tested movie-trivia items.
- Across studies, AI access raised confidence while cutting accuracy (correct answers fell from ~27.5% to ~9.2%); automatic display of AI answers produced similar effects, and modest financial incentives only partially mitigated the shift.
- The result shows that AI availability can substantially change human epistemic behavior—reducing reluctance to guess and increasing misplaced confidence—even when AI is unreliable, which matters for how AI is deployed in decision contexts.
WHY IT MATTERS
The result shows that AI availability can substantially change human epistemic behavior—reducing reluctance to guess and increasing misplaced confidence—even when AI is unreliable, which matters for how AI is deployed in decision contexts.