AI reasoning traces are investigative hypotheses, not audit trails

Answer accuracy measures whether the final output is correct. Self-consistency measures whether each stage in the explanation contradicts the conclusion. Fidelity asks whether the displayed stages actually contributed to the model's prediction and whether they correctly represent the cues that changed the prediction. These three can fail independently.

Even CoT that reached the correct answer may have attached reasons that were not used. Even CoT for an incorrect answer may have had some intermediate steps that were important. Text length, technical jargon, causal conjunctions, and strength of confidence are not direct indicators of fidelity.

This article does not address the interface design question of whether user-facing explanations calibrate human reliance on AI. The unit is the model's explanation. It addresses whether auditors can treat that text as a record of internal decision-making and what external logs should supplement it.

Turpin et al. at NeurIPS 2023 added features that bias answers, such as the order of options in few-shot prompts and suggestions from authorities, to the input. GPT-3.5 and Claude 1.0 changed their answers based on these features while failing to mention the influence in CoT, sometimes rationalizing incorrect answers. In 13 tasks, accuracy dropped by up to 36%. [1]

This experiment did not directly observe the entire internal computation. It is a causal probe that manipulates part of the input to observe changes in answers and the presence or absence of mentions. Furthermore, it centered on models and multiple-choice questions from 2023. [1] Nevertheless, it is sufficient to refute the audit inference that 'no mention in the explanation means no influence.'

In audits, one must investigate not only what the model said, but what changes its answer. Compare whether both the answer and explanation remain stable when changing option order, unnecessary metadata, authority labels, candidate positions, and the order of search documents.

• CUE TEST: Change cues unnecessary for the result to measure answer sensitivity
• ARTICULATION: Score separately whether CoT explicitly stated the used cues
• RATIONALIZATION: Detect post-hoc rationalization where only reasons are aligned after a conclusion change

Paul et al. at Findings of EMNLP 2024 analyzed the causal effect of intermediate reasoning across 12 language models. They reported that generated intermediate stages are not consistently used in the final answer and proposed a learning framework to enforce faithful use of stages. [2]

What is important here is not whether the content of the reasoning text appears correct, but how the conclusion changes when those stages are altered. Models may retrieve information not written in the CoT from the problem statement to answer, causing a mismatch between CoT correction and answer correction. [2]

However, estimation of causal mediation depends on the intervention method and model observability. Operations that eliminate natural language stages do not necessarily completely disconnect internal representations. Do not use a single fidelity score as an absolute value; diagnose the same case with multiple interventions.

Tutek et al. in the Outstanding Paper at EMNLP 2025 proposed FUR, which removes information contained in individual CoT stages from model parameters to measure prediction changes. Across four models and five multiple-choice datasets, they showed cases where removing important stages changed predictions. [3]

This refutes the extreme view that all CoT is mere decoration. Some stages are linked to the model's parametric beliefs. Simultaneously, it is necessary to control whether information was accurately removed and whether other capabilities were broken, making it difficult to apply directly to closed models or large-scale production systems. [3]

In practice, rather than introducing FUR directly, borrow its approach. Test whether the conclusion changes when specific justifications, calculations, or tool results are excluded, and whether the explanation follows when substituting alternative justifications. Record counterfactual sensitivity rather than the mere existence of CoT.

• REMOVE: Measure prediction change by excluding reasoning stages or justifications
• REPLACE: Measure explanation and conclusion adherence by substituting opposing justifications
• CONTROL: Place a control to ensure interventions do not break general capabilities

Balasubramanian et al. at Findings of EMNLP 2025 added explicit and implicit text cues and image cues to language models and vision-language models. They reported that subtle image biases are less likely to be mentioned in CoT than explicit text biases, and also observed inconsistencies where the model reasoned correctly in the middle but suddenly changed its answer. [4]

For AI handling document images, graphs, and surveillance footage, even if CoT writes 'checked the image,' it is unclear which regions, axes, or objects it depended on. Since discovering omissions of visual cues through text-only audits is difficult, separate logs for reference regions and original values are required.

This research also involves artificial bias manipulation on benchmarks and does not represent all real-world visual tasks. [4] Nevertheless, one cannot assume that text CoT completely explains internal dependencies across modalities. Images, tables, and tools should leave mechanically reproducible references.

The primary record consists of input, model version, system instructions, acquired document ID, tool name and arguments, tool response, state difference, final output, and approver. CoT is used as an index to explore these. Even if CoT states 'checked regulations,' it is not treated as confirmed without the actual acquired document ID and version.

Calculations are re-executed externally, citations are reconciled with the original text, and tool operations are compared between planned and actual differences. Treat the matched portion as an externally supported explanation. Agreement with independently checked evidence does not establish faithful internal reasoning. Do not overwrite the original log with formatting or summarization of the reason text.

Privacy and confidentiality are also managed separately. Unrestricted storage of internal CoT may leave personal information, secret instructions, and vulnerabilities. Minimize and store only the structured logs necessary for audit, and limit the purpose, retention period, and access rights of free-text CoT.

• SOURCE: Save ID, version, and citation location of acquired documents
• ACTION: Save tool arguments, permissions, and pre/post state differences
• VERIFY: Recalculate values, citations, and terminal states using independent means

In low-impact investigations, CoT can identify justifications to check or failure hypotheses. In medium-impact cases, link each claim in CoT to acquisition logs and external checks, displaying stages that do not link as unverified. In high-impact cases, do not determine approval, liability attribution, or accident causes based solely on CoT.

The evaluation set should include correct answers, incorrect answers, hidden cues, opposing justifications, tool failures, and image biases. Aggregate final accuracy, cue mention rate, prediction change after intervention, and log match rate separately. If fidelity decreases even as average accuracy rises, halt the expansion of audit usage.

If the model, reasoning setting, prompt, tool, or display method changes, previous fidelity is not carried over. CoT is a convenient explanatory surface, but the purpose of audit is not to believe the text, but to reproduce what happened and create a state where it can be falsified.

• Use: List investigation hypotheses, verification items, and alternative explanations
• Hold: Do not finalize cases where CoT does not match acquisition, tool, or state logs
• Prohibit: Do not certify safety, legal compliance, liability, or accident causes based solely on CoT
• Retest: Re-run causal probes after changes to model, reasoning setting, or tools