AI Security

AI security protects AI systems and their assets against unauthorized access, manipulation, and disruption. Its scope includes ordinary application security and attacks involving learned behavior or model inputs.

Accuracy is not protection

A fictional assistant can retrieve the correct refund amount for the wrong customer. Its answer is accurate, yet disclosure violates the intended access boundary. Another request may consume all available workers without exposing any data.

Protect records, credentials, source documents, model artifacts, and service capacity according to their role.

Follow the whole system

Check ingestion, deployment, retrieval, tools, rendered answers, caches, and logs. A model's refusal does not prove that a preceding tool call was blocked. Observe what data crossed the boundary and what action occurred. Model instructions can support controls, but authorization and resource limits need enforcement in the services doing the work.

Reference: NIST: Information Security.