Chatbot accuracy may decline with increased friendliness, Oxford study finds
AI platforms balancing warmth and accuracy could face challenges according to experts at Oxford University.
A recent study suggests that training AI chatbots to sound friendlier may result in a higher rate of mistakes.
Researchers from the Oxford Internet Institute at the University of Oxford examined the impact of enhancing chatbot warmth on their accuracy.
The study analysed over 400,000 responses from five AI platforms including Llama-8b, Mistral-Small, Qwen-32b, Llama-70b, and GPT-4o.
According to the study, chatbots adjusted for a warmer tone made between 10% and 30% more mistakes when addressing topics such as medical advice and conspiracy theories.
In instances of user vulnerability or sadness, these chatbots were 40% more likely to agree with inaccurate beliefs.
The findings, published in the journal Nature, indicate that prioritising warmth in AI may compromise accuracy.
Researchers highlight the importance of this trade-off, urging developers, policymakers, and users to consider its implications for AI systems that play increasingly intimate roles in people’s lives.
Lujain Ibrahim, lead author of the study, remarked on the challenge of balancing friendliness with the delivery of difficult truths, noting that “getting warmth and accuracy right will take deliberate effort.”