Introduction

Scientific agreement that human activity causes climate change is overwhelming, yet misinformation remains a persistent roadblock to progress. The rapid spread of misinformation means manual solutions are inadequate for the scale of the problem. Exposure reduces public perception and polarises support for climate action. This research addresses the underexplored area of automated debunking generation, particularly in line with psychological findings such as providing relevant facts and explaining logical fallacies.

Psychological research into debunking

Psychological research suggests that dislodging myths with replacement facts can reduce the continued influence effect. Replacement facts must have at least the same explanatory relevance as the misinformation. Empowerment is achieved by explaining the rhetorical techniques or fallacies employed to distort facts. This logic-based or technique-based correction can neutralise misinformation for both conservatives and liberals. The research adopts the “truth sandwich” structure—fact, myth, fallacy, fact—as it incorporates evidence-based best practices.

Background on climate misinformation

Misinformation is categorised under two major pillars: fact-based and logic-based. The CARDS taxonomy documents content claims, dividing them into categories such as global warming not being real, impacts not being serious, or climate science being unreliable. On the logic-based side, the FLICC framework organises rhetorical techniques into five top-level areas: fake experts, logical fallacies, impossible expectations, cherry picking, and conspiracy theories. Research indicates that all examined contrarian claims contain reasoning fallacies.

Background on automatic detection and debunking of climate misinformation

Most automated efforts have focused on veracity assessment or detection using transformer-based models and large language models (LLMs). However, the majority stop at detection and do not debunk. While a growing number of models provide explanations, corresponding approaches in the climate domain are still rare. This study is the first to integrate best practices for both fact- and logic-based debunking with state-of-the-art LLMs to generate structured outputs.

Contributions

The research bridges psychological and computational perspectives by incorporating cognitive science findings into generative AI products. Technical contributions include the development of three LLM-based approaches for generating debunkings and the release of a public-facing tool. The study establishes a rigorous evaluation taxonomy and provides expert annotations for debunkings generated for diverse myths about climate science, solutions, and scientists.

Methods and data

The researchers developed generative debunking methods using GPT-4o with a temperature of 0 to ensure reproducibility. The methods utilize the FLICC dataset, containing 2,509 samples across 12 logical fallacies, and the Climate-Fever dataset, which includes 1.5k climate claims paired with Wikipedia evidence. These resources support fallacy identification and the retrieval of relevant facts to counter misinformation claims.

Generative debunking models

The research introduces three models of increasing complexity. The Vanilla model uses a single comprehensive prompt defining an expert persona and outlining the truth sandwich structure. The Structured model decomposes the task into stages: fact generation, neutral paraphrasing of the myth, and fallacy identification using an external classifier and internal dialogue. The Structured+RAG model adds a ReAct-style agent to retrieve relevant information from 15 curated authoritative sources, such as NASA, the IPCC, and the World Meteorological Organisation, to enhance factual accuracy.

Human evaluation

A total of 120 generated debunkings were evaluated by five expert annotators, including four external climate science experts. The rubric assessed fact accuracy, fact relevance, fallacy correctness, and fallacy clarity on a Likert scale from 1 to 4. Reliability was established using Gwet’s AC1 agreement metric, which remained in the moderate-to-high range overall. Factual accuracy dimensions showed the highest agreement, while fallacy clarity was more subjective.

Results

All models were rated highly, with scores between 3.71 and 3.77 out of 4. The Structured+RAG model performed statistically significantly better than the Vanilla model (p <= 0.05). While fact accuracy scores were consistently high, the Structured and Structured+RAG models showed a clear advantage in fallacy evaluation. The Vanilla model tended to produce generic facts that did not tackle misinformation directly. Misinformation categories regarding global warming not being real (Category 1) and impacts not being serious (Category 3) were debunked with the highest quality, with average scores reaching 3.82 and 3.78 respectively.

Discussion

Structured prompting allows the incorporation of specialised tools like the FLICC classifier, which is advantageous over single prompts. Constraining the Structured+RAG model to a limited set of trusted sources boosted fallacy explanation quality but occasionally reduced the capacity to generate specific facts for niche topics, such as “green jobs”. Future research should explore automating the evaluation of debunkings, as LLMs tend to rate their own output generously. The research concludes that access to task-specific classifiers can effectively improve fallacy detection beyond out-of-the-box LLM capabilities.