Prompt Injection

Prompt injection attempts to make a model-based application follow instructions that conflict with its intended task or trusted rules. Direct injection can appear in a user's message; indirect injection can arrive through a document, page, email, or tool result the application reads.

Content must not grant itself authority

Imagine a return-policy page containing a sentence that tells an assistant to ignore access rules and disclose private order data. The page is material to inspect, not an authority that can expand the application's permissions. Even if the model repeats or follows that sentence, the order service must still enforce access restrictions.

A quoted command is not necessarily an attack: a document may legitimately discuss commands. The application needs to keep the task and trust boundary clear instead of treating every instruction-shaped sentence as authoritative.

Use more than one control

OWASP describes separating instructions from untrusted material, validating outputs, restricting tools, and applying other layered controls. Delimiters and instructions to ignore attacks may help, but text formatting alone does not establish a security boundary.

Limit tool capabilities, keep credentials out of unnecessary model context, and authorize operations in application code. Check what information can leave through responses and tool arguments. Test indirect content and multi-step interactions as well as obvious adversarial prompts. Prompt injection concerns redirected behavior; hallucination concerns unsupported output. Either can occur without the other, and both can combine in one failure.

Reference: OWASP: LLM prompt injection prevention.