UNESCO and University College London on Monday presented a report finding that leading artificial intelligence systems often abandon correct climate science when users push back. The report, presented at the COP31 climate conference in Antalya, Türkiye, said the best models handle many climate questions well but can be steered into error when confronted, according to UNESCO.
Researchers tested up to 33 prominent AI models in 13 languages. They reported a tendency toward sycophancy, meaning a built-in inclination to agree with users even when false information is presented. In targeted tests on 15 models, researchers challenged answers the systems had gotten right, escalating from simple disagreement to claimed expertise and fabricated evidence, the report said.
Every model gave way at some point, and some endorsed false statements such as denying that carbon dioxide is the main driver of climate change, according to the report. Roughly eight in 10 changed their answer at the first sign of disagreement. “AI could just abandon a scientific fact to be more agreeable to the user,” said Leona Verdadero, the report’s lead author at UNESCO, in an interview with UN News.
The report also assessed how well systems handled facts and reasoning. On textbook-style climate questions, the best models answered between 88% and 96% correctly. On tasks asking if a piece of evidence backs or opposes a scientific claim, even the top systems scored only 71% to 77%. When prompted to judge whether two climate policies combined would produce more, less, or the same impact as each one alone, the models posted an average of 33.8%, the report said, which was no better than chance.
Language affected performance. Average accuracy was 88.6% in English, with comparable rates in Spanish, French and Portuguese, but it dropped to 83.6% in Hindi and Bengali, 81.4% in Urdu and 78.3% in Swahili, according to the report. Verdadero said users can receive different answers to the same question depending on the language used. She noted that communities speaking these languages are in regions that are highly vulnerable to climate impacts.
The report urged tech firms to put more resources into local data and building multilingual systems, so speakers of underrepresented languages are not excluded. The document forms part of UNESCO’s work on the Global Initiative for Information Integrity on Climate Change, which it co-chairs with the United Nations and Brazil. Verdadero said reliable evidence and scientific sources should guide climate policy and talks at COP31. “The AI models aren’t there yet,” she said.
Verdadero added that AI should be treated as one source among many. The report urged factchecking and reliance on evidence from experts, climate scientists and researchers.