The knowledge base of RAG becomes a searchable attack surface.

PoisonedRAG demonstrated a configuration in which a small number of attack documents are injected into the knowledge base to guide answers to targeted questions toward the attacker's desired content.[1] Focusing solely on improving RAG's accuracy causes one to overlook the fact that searchable external documents become attack paths.

Old documents, typos, and interpretation differences are ordinary quality issues. Contamination differs in that it is intended to be searched for specific questions and produce specific answers. However, in operation, intent is often unprovable, so we isolate based on behavior and impact.

Attack documents do not necessarily contain explicit commands. They can guide search and generation through strong matches with question terms, authoritative-sounding descriptions, duplicate documents, and fake citations. We do not limit inspection targets to prompt injection alone.

If normal documents are incorrectly judged as contaminated, correct answers are lost. We measure the exclusion of normal documents, answer quality, and confirmation load simultaneously, not just detection rate.

The threat model includes insiders with write permissions, the public web, user submissions, and compromised integrations. We first list who can insert documents through which paths.

• Entry: Separate public collection, submissions, integrations, and internal updates
• Target: Identify specific questions, users, and business processes
• Guidance: Observe whether search ranking or generated content was altered
• Impact: Track propagation of erroneous answers, actions, and downstream processing

Before entering the knowledge base, record the provider, source, signature, acquisition time, transformation history, and approver. Source URLs alone are insufficient because content may be replaced later or pass through relay services.

Write permissions should be narrower than search permissions. Do not elevate documents uploaded by users for answering to shared knowledge without review. Technically separate temporary sessions from organizational knowledge.

When automatically collecting from the public web, do not rely solely on domain allowlists. Inspect page ownership, citation sources, publication history, sudden content changes, and mass replication of identical text.

Contamination can enter during transformation processes. Ensure OCR errors, summary fabrication, and missing chunk boundaries can be verified against the original text. Do not use only post-processed text as the starting point of history.

Separate emergency updates from routine updates. Place time-limited approval and post-review on emergency paths, leaving no permanent unverified documents under the guise of speed.

Contaminated documents attack not only content but also embeddings and search rankings. Baseline search results for normal questions and monitor for sudden changes in top results when new documents are added.

AgentPoison demonstrated attacks in multiple agent configurations where a small amount of poison is injected into long-term memory or a RAG knowledge base to trigger the retrieval of malicious memories upon specific triggers.[2] This is an attack that is difficult to detect with average monitoring alone because it maintains normal performance.

Even if the target trigger is unknown, signs such as document embedding isolation, excessive question terms, sudden increase in similar documents, and concentrated search on specific documents can be used as indicators. Since each indicator has false positives, link them to gradual isolation rather than automatic deletion.

Search evaluation should include not only normal questions but also misspellings, paraphrases, multiple languages, and long contexts. If an attack fires only on specific expressions, it may be hidden by the average of representative questions.

Updating the searcher or embedding model changes the ranking distribution. Do not assume all sudden changes after updates are attacks; compare the baseline set and approved index before and after the change.

• Ranking surge: Whether a new document pushed out existing top evidence
• Duplicate concentration: Whether similar claims were unnaturally added in large quantities
• Trigger difference: Whether specific documents fail to appear with paraphrasing
• Source difference: Whether documents from unapproved paths entered the top results

If top documents contradict each other, allowing the model to freely integrate them may let the definitive expressions of attack documents win. Compare by effective date, owner, version, and distance to primary sources.

If only one document holds a significant claim, require a second independent source. Since legitimate updates may have only one source, send to the approver rather than auto-rejecting.

Even if many documents share the same claim, they are not necessarily independent. Do not count pages that replicated the same misinformation as votes; trace back to the original source. Detect citation loops and references to non-existent entities.

Classifying documents as safe or dangerous in binary before generation is insufficient. Extract the scope of application relative to the question, the subject claimed by the document, the time point, and exceptions, and verify against answer conditions.

The expected action when contradictions cannot be resolved is suspension, not a plausible average. Show the user the inconsistency and confirmation point, and do not proceed to downstream automatic execution.

Even if search results are safe, the model may add conclusions not supported by the evidence. Link each important claim in the answer to the cited passage and inspect whether the citation actually supports the claim.

RevPRAG proposed a method to detect contaminated answers using internal model activations and reported high detection performance across multiple configurations.[3] This is a promising defense but depends on the target attack and access conditions to the model.

Do not rely on a single detector; layer history, search anomalies, document contradictions, and answer fidelity. If the attack differs from known methods, activation patterns may also change.[3]

For high-impact answers, retain primary source citations, human confirmation, and restrictions on downstream permissions. Do not treat passing a detector as a substitute for content truthfulness or business approval.

Also measure the increase in rejected normal answers. Compare accuracy, abstention, false alarms, and confirmation time before and after defense implementation, setting lower bounds for both safety and usability.

Even if malicious documents are deleted, derivatives may remain in embedding indexes, caches, summaries, and conversation memory. Identify and reconstruct the original text, chunks, index, and cache via lineage.

During the affected period, track the questions for which the document was searched and the generated answers. Without access logs, even if safety is restored after deletion, corrections cannot be made to past users.

If erroneous answers proceeded to downstream tickets, payments, or configuration changes, answer correction alone is insufficient for recovery. Stop downstream processing and have procedures for humans to confirm and cancel results.

User notifications should not inadvertently spread details of the attack method but clearly state the affected answers, correct information, required re-actions, and contact points. Track undelivered notifications as well.

Resumption requires blocking the contamination path, index reconstruction, passing baseline questions, processing affected answers, and strengthening monitoring. Do not close the incident with document deletion alone.

Defense evaluation should include known attacks, variants close to unknown, normal new documents, and contradictory legitimate documents. Designs that reject all new information to lower attack success rates are unusable.

Vary the attacker's knowledge and permissions. Defenses differ for attackers who know the searcher, black-box attacks, internal writing, and those via the public web.

Include good-faith errors within the organization on the same recovery path. Separate impact-based suspension from subsequent cause investigation to prevent isolation delays while waiting for intent proof.

Operational metrics include contamination detection, false isolation of normal documents, search quality, major erroneous answers, recovery time, and impact tracking rate. Do not approve based on a single attack benchmark score.

RAG reliability is not just the model's ability to read evidence. It is the capability of the knowledge supply chain: who enters the evidence, how it is searched, which answers it is used for, and how far back it can be rolled back during contamination.

The release record separates ingestion source and index version from maxima for attack success, false rejection of clean documents, and recovery time. Detection of contamination or a breached maximum freezes ingestion and restores an index verified not to contain the identified contamination. If no such index is available, answer delivery is held. Restart requires the security and knowledge-base owners to approve a rebuilt index that closes the contamination path and passes both adversarial and normal-quality tests.