AI-text detectors cannot stand alone as evidence for adverse decisions

The first question is whether the text has statistical features characteristic of a specific model. The second is attribution regarding which generation system created it and when. The third is a policy judgment on whether AI use exceeded permitted scope or the author's intellectual contribution was insufficient. Detectors can potentially answer mainly the first question, and these three must not be collapsed into a single score.

Texts proofread by AI, texts where humans extensively revised AI drafts, translation assistance, and human texts with many formulaic phrases involve a mix of generation, editing, and authorship. "AI-like" is not synonymous with "created solely by AI." Quality, plagiarism, and factual errors must also be investigated separately.

This article does not address provenance management for images and videos such as C2PA. The focus is primarily on post-hoc text detection and generation-time watermarks, and how to connect them to scoring, discipline, and publication decisions. Final legal and disciplinary judgments are left to individual regulations and due process.

Sadasivan et al. stress-tested multiple methods, including neural detection, zero-shot detection, search-based detection, and watermarks, using recursive paraphrasing in TMLR 2025. They showed conditions under which detection rates could be reduced without significantly lowering semantic quality in texts of approximately 300 tokens, and demonstrated the possibility of spoofing where human texts are mistaken for AI texts. [2]

These results do not mean that all detectors are always invalid. There are methods that can distinguish texts for specific models, at specific times, and in unmodified texts. The problem is that while closed evaluations at deployment may show high performance, the distribution changes in production when users paraphrase, translate, summarize, and unknown models increase. [2]

Therefore, if evaluating detectors, include not only unmodified texts for known models but also unknown models, human editing, multiple paraphrases, short texts, specialized texts, and non-native language texts. Instead of only increasing the true positive rate, measure who is wrongly suspected at the same threshold.

• UNKNOWN MODEL: Include generators and versions not seen during training in external tests
• TRANSFORM: Test after translation, summarization, proofreading, and multiple paraphrases
• SHORT TEXT: Show uncertainty by text length and do not make definitive judgments on short texts

Kirchenbauer et al. presented a watermarking method in ICML 2023 that favors a subset of next-token candidates based on a secret key, embedding statistical signals into the text. It is designed to be detectable with corresponding tests without accessing the model's API or weights. [1] This holds under conditions where the generator cooperates and can control the probability distribution during generation.

Watermarks make it easier to attribute outputs to specific providers compared to detectors that infer origin solely from style. However, they require sufficient text length, secret key management, decoding settings, and resistance to modification. Providers without watermarks, texts where humans wrote first and then AI proofread, or external models from other providers cannot be judged by that watermark. [1]

The detection of a watermark is not proof of misconduct. It may be permitted assistance, templates provided by the organization, or accessibility support. Conversely, the absence of a watermark is not proof of human creation. It is important to specify the applicable population and not treat non-detection as a get-out-of-jail-free card.

Liang et al. reported in 2023 that when comparing seven public detectors on TOEFL essays and essays by native English speakers in the US, the rate of misjudging human texts from non-native speakers as AI-generated was high. The average misjudgment rate for non-native speaker essays was 61.3%, compared to 5.1% for native speaker essays. [3] Methods that equate the predictability of style with "AI-likeness" may have confused it with vocabulary breadth.

On the other hand, Jiang et al. in 2024 constructed a specialized detector using language features and perplexity on properly sampled large-scale GRE essays and found no evidence of bias against non-native speakers. [4] This does not invalidate prior results but shows that fairness and accuracy change depending on training data, target exams, detectors, and populations.

Combining the two studies, we cannot say that "AI detection always discriminates against non-native speakers" nor that "fairness is solved if accuracy is sufficiently high." Continuously measure false positives by language background, text length, and task using the versions and local text populations in use. [3][4]

• FALSE POSITIVE: Measure the rate of suspecting human texts by language background, text length, and task
• FALSE NEGATIVE: Measure the rate of missing paraphrased AI texts at the same threshold
• ABSTAIN: Establish an uncertainty zone and do not force a binary choice

AUC and mean precision do not directly indicate the evidentiary value needed for discipline. In populations with low AI use, even if false positives are few, many positive judgments may be human texts. In operation, set the AI usage rate corresponding to the assumed prevalence and calculate the positive predictive value and number of false accusations.

Thresholds vary by use case. For low-stakes editing assistance, high recall may be acceptable, but for disqualification, dismissal, or public suspension, the loss from false positives is large, and detectors cannot be used beyond the entry point alone. Confirm the possibility of remediation arising from the same disciplinary threshold across groups, not just average differences.

Performance degrades over time. Record model versions, detector versions, language, task, text length, and modification methods, and update blind samples at regular intervals. If a vendor changes the internal model, do not carry over past passes; re-evaluate.

In investigations, save the task, permitted scope, submission time, version history, citations, references, and work logs before the detection score. Next, show the text segments that triggered the detector and the uncertainty, and provide the author an opportunity to explain the tools used, editing process, and basis. Do not treat failure to explain as an automatic confession.

Passing the same text through multiple correlated detectors does not necessarily increase independent evidence. Detectors using the same training data or perplexity features repeat the same errors. Combine evidence with different failure modes, such as watermarks, editing history, source verification, and oral reproduction.

Reviewers should be able to confirm the original text and evidence bundle while blinded to the initial score and disciplinary proposal. If the detector version or threshold was changed afterward, re-evaluate similar cases, and if misjudgments are found, correct not only the individual case but all cases under the same conditions.

• CORROBORATE: Seek two or more pieces of evidence with different failure modes
• DISCLOSE: Show the detector version, threshold, uncertainty zone, and known limitations to the author
• APPEAL: Provide a pathway for independent human re-review and correction of similar cases

For low-stakes uses, detectors can be used for self-checking or additional confirmation. For medium-stakes, limit to triage for starting version history or source verification. For high-stakes, prohibit discipline based solely on detectors and do not finalize adverse actions until appeals are resolved.

Use watermarks as attribution evidence only when the provider has officially implemented them, manages keys and versions, and the target model is clear. Hold in cases of key leaks, text modifications, short texts, or incompatible models. Do not use the absence of watermarks as proof of human creation.

Suspension conditions include increased false positives in external tests, language group disparities, decreased performance on unknown models, increased cancellations via appeals, and unexplained vendor version changes. Resumption requires not only performance improvement but also the re-evaluation of past cases, notification to authors, and completion of corrections. The purpose of detection is not to find culprits but to prevent harming people due to insufficient evidence.

• Use: Limited to low-stakes additional confirmation and discovery of missing provenance
• Hold: Do not draw conclusions for short texts, unknown models, mixed editing, or uncertainty zones
• Prohibit: Do not finalize disqualification, discipline, admission exclusion, or public suspension based solely on detection scores
• Resumption: Return thresholds only after external re-validation and correction of past misjudgments are completed