Poisoning and Evasion

Poisoning manipulates data or model preparation to affect learned behavior. Evasion manipulates an input presented to a deployed model to induce an unwanted prediction. They differ in the stage the attacker influences.

Locate the intervention

In a fictional classifier, altered training labels could teach an incorrect association. A modified document submitted only during prediction could instead cause a particular misclassification without changing model parameters.

Record which data, artifact, or input the attacker can control. That determines which protections and tests are relevant.

Retrieval corruption is another path

Changing a RAG document can distort retrieved evidence without retraining a model. It may be described as retrieval-data poisoning, but should not be confused with changing learned parameters. Protect source editing and indexing as well as training. A wrong prediction alone does not identify which attack, if any, caused it.

Reference: NIST: Adversarial Machine Learning Overview.