Conversational-AI corrections must be measured through later memory and behavior

The purpose of correction is not to move the other party to an attitude desired by the organization. It is to provide more direct and current evidence for identifiable factual claims, replacing incorrect causal explanations or sources. We do not assign a single truth label to value judgments, predictions, or personal experiences.

Measurement also requires more than just a decrease in agreement immediately after the conversation. We must distinguish whether the new explanation can be correctly reproduced, whether it persists weeks later, whether related behaviors change, and whether there are side effects such as doubting correct information. Satisfaction and knowledge, and knowledge and behavior, are distinct outcomes.

This article does not certify the truth or falsity of individual claims in politics, health, etc. It also differs from persuasion optimization for sales or solicitation. After fixing the correction target and primary source externally, we ask whether AI can be used for dialogue adaptation and explanation generation.

Boissin and colleagues’ preregistered PNAS Nexus 2025 experiment analyzed 955 participants after applying preregistered exclusions to 1,997 completers. Everyone completed a two-round, evidence-based GPT-4o dialogue, and beliefs decreased from before to after the conversation. Randomization varied the AI-versus-expert speaker label and tone; differences between those conditions were not statistically detected.[1] There was no untreated group. The explanation that message content is the main factor is the authors’ interpretation.

An additional reduction in belief attributable to the AI label was not statistically detected in this experiment. This does not prove that labeling has no effect or that the same result holds in other settings. While the ability to align refutations with the other party's specific claims is promising, participants consented to the study, described their own beliefs, and directed attention to a short dialogue. [1] The same effects cannot be transferred to natural SNS use, repeated exposure, or executive action.

This research overlaps with the preceding Science research, which has publication concerns mentioned later, in terms of authors and research lineage, but is a pre-registered experiment with a different sample. [1][5] It is not counted as replication by an independent research team and is treated as additional evidence within the same lineage.

• CLAIM: Specify one factual claim to correct
• EVIDENCE: Fix current primary sources, not the model's internal knowledge
• OUTCOME: Measure agreement, understanding, retention, and behavior separately

The AAAS-deposited Crossref record confirms that Science published an Editorial Expression of Concern on 11 June 2026 for the 2024 article “Durably reducing conspiracy beliefs through dialogues with AI.” [5] The checked evidence is the registered metadata and its link to the original article; the reasons in the notice body and the publisher’s final disposition remain unverified.

This registered record alone does not establish whether the publisher accepted corrected author analyses or removed the concern. [5] This article does not use the original claims of approximately 20% or persistence over two months as positive evidence. Neither retraction nor a finalized result is inferred from the existence of the concern.

The practical lesson is to continuously verify the version used, corrections, expressions of concern, and analysis code, regardless of whether the journal is prestigious. Place the same gate on the research cited by correction AI, replace the answer basis if the publication status changes, and re-evaluate past correction deliveries.

Suzgun et al. evaluated 24 models on 13,000 questions across 13 epistemic tasks in Nature Machine Intelligence 2025. Even with new models, performance dropped significantly on tasks requiring confirming first-person false beliefs as 'that person believes it,' confusing fact correction with belief attribution. [2]

In correction dialogues, it is necessary to distinguish whether the other party's utterance is a factual claim, hearsay, hypothesis, anxiety, or irony. If the model simply denies first-person false beliefs, it impairs conversational understanding. Conversely, if belief confirmation is confused with agreement with facts, it reinforces misinformation. [2]

KaBLE is an artificial benchmark and does not measure the rate of erroneous corrections in real, long conversations. [2] Nevertheless, it is insufficient to simply instruct the model to 'find and correct misinformation,' leaving target selection and evidence verification to it. Humans must first classify claims and epistemic states.

• STATUS: Classify facts, beliefs, citations, hypotheses, and value judgments
• SCOPE: Limit correction targets to parts that can be refuted by evidence
• HUMILITY: Withhold judgment and do not generate assertions if evidence conflicts or is insufficient

Clark and Lewandowsky conducted three pre-registered experiments in Communications Psychology 2026 using deepfake videos. While specific warnings reduced the impact, some participants who believed the warning and understood the video was fake still used the video content for determining a person's guilt. [3]

Warnings partially reduced influence. In Experiment 2, the specific warning increased recognition that the video was fake more than the generic warning, but the difference between the warnings in reducing guilt judgments was not statistically detected. Warnings appeared before viewing, and video influence remained among participants who accepted them.[3] The experiment did not directly test post-exposure correction or alternative explanations. As an operating hypothesis, compare evidence-based alternative explanations with denial labels in separate trials.

The research used fictional characters and nested fiction, so it is uncertain whether the reactions are the same as in reality. [3] Nevertheless, this provides a reason not to use label display rates as a success metric, but to measure whether incorrect content continues to be used in judgments after correction.

Maertens et al. conducted five pre-registered longitudinal experiments with a total of 11,759 participants in Nature Communications 2025. There were conditions where text and video interventions teaching misinformation tactics one month prior remained effective, and additional interventions that triggered memory suppressed decay. [4]

This research is not a correction test for conversational AI, but a study on psychological inoculation and memory. [4] Therefore, the effect size cannot be directly transferred to AI dialogues. The implication for practice is not to consider correction complete after sending it once, but to measure the timing of memory, re-exposure, and additional explanations.

Google Jigsaw-affiliated authors contributed, and Google Jigsaw directly funded the study and some authors. The paper declares no competing interests.[4] Record these organizational and financial roles. As an operating proposal, separate intervention implementers from evaluators and independently test viewing and sharing behavior on platforms.

• IMMEDIATE: Measure immediate beliefs and understanding of alternative explanations
• RETENTION: Measure reproduction and identification after a delay
• BEHAVIOR: Measure actions corresponding to the target, such as sharing, purchasing, or voting
• SIDE EFFECT: Measure over-distrust of correct information and general distrust

For correction candidates, humans approve the claim ID, original text, speaker, context, current primary source, source date, and the scope of refutability. AI may vary the order of explanation and examples to match the other party's reasons, but does not change the set of evidence or the scope of the conclusion. If new facts are generated, they are re-reviewed before transmission.

In trials, we compare static correction, personalized AI dialogue, human dialogue, and no intervention. We mix correct information with misinformation to measure whether AI erroneously corrects correct claims. Users can choose to rebut, withhold, or switch to a human, and a list of evidence is provided after the conversation ends.

Stop conditions include non-existent sources, erroneous correction of correct claims, deterioration of publication status, backlash from specific groups, and over-distrust of correct information. If an erroneous correction is found, the conversation is not hidden; a correction is issued to all subjects who received the same evidence version, and the search index and templates are invalidated. Making the correction mechanism itself retractable is an implementation condition.

• Adoption: Understanding and retention improve within the scope fixed to primary sources
• Limitation: Withhold value judgments, predictions, and evidence conflicts, and hand over to humans
• Stop: Detect erroneous corrections, fictional sources, deterioration of publication status, and over-distrust
• Resume: Replace the evidence version, complete corrections and re-testing for past subjects