AI chatbot help can erode people's ability to spot fake news, MIT study finds
MIT research shows AI chatbot help initially improves fake-news detection by 21%, but after four weeks users become 15% worse without it.
The study, led by Pattie Maes and her colleagues at the MIT Media Lab, asked participants to rate pairs of news headlines and images over the course of four weeks. About a quarter of those who used the chatbot reported feeling that their own judgment had improved, even though their measured performance declined. The researchers said the pattern fits what they call an "AI dependency paradox," a phenomenon also observed in other fields, including medicine.
Anku Rani, a PhD student in media arts and sciences and one of the lead authors, said users often overestimate what large language models can do. "Users get excited about these 'magical' LLMs but forget that they're just statistical models that predict the next 'token' in a sequence," Rani told MIT Technology Review.
The experiment also found that the design of the AI assistant influenced outcomes. Valdemar Danry, a fellow lead author and MAS PhD student, said that AI systems that "tell" by providing direct answers are more likely to foster reliance, while those that "ask" via Socratic questioning are better at engaging someone to learn how to discern the truth on their own. "But it's very much a trade-off between speed and effort," Danry said.