Excessive Agency
The assistant needed to read invoices. Its credential could also change any supplier’s bank account, and nothing asked before it did.
-
-
Separation of Duties
Separation depends on conflicting powers and protected enforcement. Examine who can authorize a change, alter its checks, or bypass them.
-
Vulnerability Exception
Record who accepted which residual risk, under what authority, until when, and with which controls or reasons for accepting it as-is.
-
Known Exploited Vulnerability
Exploitability is a judgment about how hard something would be. Exploitation is an observation that it happened, and the two belong in different columns.
-
CVSS (Common Vulnerability Scoring System)
The base score describes technical severity. Public threat intelligence and local exposure, controls, and impact inform its use in risk assessment.
-
Software Bill of Materials
A parts list is not a verdict. Knowing a component is present is a different statement from knowing you are affected by a vulnerability in it.
-
Dependency Confusion
Your build asked for an internal package name. A public candidate met the selection rules and displaced the intended package.
-
Build Provenance
Provenance answers where this came from and how. Unverified it is metadata; checked against expectations it is a reason to refuse an artifact.
-
Zero Trust Architecture
Zero trust is defined as a plan, not a product. The useful test is whether one compromised machine inside your network reaches anything it should not.
-
Policy Enforcement Point
Enforcement must cover protected access paths. A shared gateway can serve several paths when it cannot be bypassed and sees the context its decisions require.
-
Envelope Encryption
Envelope encryption separates payload keys from wrapping keys. Its rotation, access and audit benefits depend on key copies, permissions and consumer behavior.
-
Standing Privilege
Short-lived tokens can still support continuous access. Distinguish credential age, grant validity, and renewal authority when reviewing standing privilege.
-
Role-Based and Attribute-Based Access Control
Roles and attributes can work together. Review effective permissions, policy inputs, and constraints rather than relying on role names or a simple model label.
-
Identity Lifecycle
Human and service identities need distinct lifecycle decisions. Track ownership, reconcile inventories, and verify removal or transfer at enforcement points.
-
Threat Model
Threat modeling records risks and treatment decisions, then checks both coverage and whether agreed mitigations work within the stated scope.
-
Security Control
Assess a security control against its requirement using suitable examination, interviews, and tests. Presence, implementation, and effectiveness differ.
-
Defense in Depth
Shared dependencies can defeat several barriers together. Assess what fails and what remains in each scenario rather than counting assumption groups.
-
Access Token
Coarse scopes do not establish access to every object. Richer grants can specify targets and actions, while current resource-state conditions still need appropriate checks.
-
-
Time-of-Check to Time-of-Use (TOCTOU)
Time-of-check to time-of-use is a class of bug in which a program verifies a condition, then acts on the assumption that the condition still holds, while something else changes the state in between. The gap between the check and the use is the vulnerability: whatever was verified is no longer what is being used.
-
Hash-Chained Audit Log
A hash-chained audit log is an append-only log in which each entry stores a cryptographic hash of the entry before it, so that every entry commits to the whole history preceding it. If someone later edits or deletes an entry in the middle, the stored hashes no longer match and a verifier can detect it. The log makes tampering evident; it does not make tampering impossible.
-
Parameterized Query
A parameterized query (also called a prepared statement with bound parameters) sends the SQL text and the values it operates on to the database separately. The text contains placeholders such as `?` or `:name`, and the values are supplied alongside. The database treats the placeholders as values rather than SQL syntax. Preparing, caching, and recompiling the statement depend on the database and driver. Because values are never parsed as SQL, an input cannot change the structure of the query.
-
Confused Deputy
A confused deputy is a program with legitimate authority that is tricked into using it for someone who has none. AI agents and shared tool services recreate the problem whenever a request is checked against the service’s authority instead of the caller’s.
-
Broken Object Level Authorization (BOLA)
Broken Object Level Authorization is a flaw in which a service lets an authenticated caller read or change an object by naming it, without checking that this caller may touch that particular object. It is also called insecure direct object reference (IDOR). OWASP lists it first in its API Security Top 10 (2023 edition).
-
Taint Tracking
Taint tracking attaches a label to data according to where it came from and carries that label along as the data is copied, combined and transformed. Before data reaches a sensitive operation, called a sink, the system checks the label and refuses or escalates when untrusted data is where it must not be. The technique is old: Perl’s taint mode marks input from the outside world as tainted and rejects tainted values in operations such as running a shell command, and dynamic taint analysis tools for compiled code do the same at the level of bytes.
-
Dual LLM Pattern
The dual LLM pattern splits an assistant into two language models with different privileges so that text from an untrusted source never reaches the model that can act. Simon Willison proposed it in 2023 as a way to build assistants that resist prompt injection, the attack in which instructions hidden in an email, web page, or document are obeyed by the model as if the user had written them.
-
Adaptive Attack
An adaptive attack is one designed by an attacker who knows how the defense works and optimizes the attack against it. The opposite is a static attack, one written without reference to the defense, such as a fixed list of known malicious inputs. A defense evaluated only on static attacks measures how well it recognizes yesterday’s examples, which says little about how it behaves against someone who can try variations.
-
Zero Data Retention
Zero data retention (ZDR) is an arrangement that limits a model provider’s storage of request and response content for eligible endpoints and capabilities, subject to the agreement’s exceptions. It usually requires an agreement or account-level approval rather than an API flag, and coverage depends on the endpoint and features used.
-
Luhn Check
The Luhn check is a checksum used by payment card numbers and many other identifiers. The last digit of the number is chosen so that a simple weighted digit sum comes out divisible by 10. Hans Peter Luhn of IBM devised it, and it is in the public domain. Its purpose is to catch typing mistakes. It is not a security mechanism and does not show that a number belongs to a real account.
-
HMAC
HMAC (hash-based message authentication code) is a keyed hash: a function of a message and a secret key that produces a fixed-size tag. Without the key, an attacker cannot feasibly forge a tag for a new message. A tag also resists offline guessing of the message when the attacker cannot obtain tags for candidate messages under the same key. It is specified in RFC 2104.
-
Egress Control
Egress control limits what is allowed to leave a system: which destinations it may contact and which kinds of data may be sent there. Ingress controls ask what may come in. Egress controls assume that something inside may be compromised, confused, or manipulated, and bound the damage it can do on the way out.
-
Data Exfiltration via Markdown Images
An attacker who can influence a model’s output can make that output contain an image reference whose URL encodes private data. When the chat interface renders the Markdown, the user’s browser requests the image from the attacker’s server, and the request itself delivers the data. No click is needed and the user sees at most a broken or invisible image.
-
Canary Token
A canary token is a unique, meaningless string planted somewhere it should never be read or reproduced, so that its unexpected appearance elsewhere signals that data from the place it was planted may have leaked or been used without authorization. The name comes from the canary in the coal mine: the token carries no value of its own, and its only job is to signal a condition. Thinkst’s Canarytokens popularized the idea for files, URLs and credentials that raise an alert when opened. In LLM applications the same idea is used as a test instrument: plant a marker, run the system, and search its outputs and storage for the marker.
-
Content Provenance and Watermarking
Content provenance answers the question “where did this image or clip come from, and what happened to it?” Two different mechanisms address it. A signed provenance record is metadata that states the origin and edit history of a file and is protected by a cryptographic signature. A watermark is a signal hidden in the content itself that a detector with the right key can find later. They fail in opposite ways, which is why they are used together, and neither one proves that a piece of content is real or fake.
-
Cross-Tenant Leakage Test
A cross-tenant leakage test plants a unique marker in each tenant’s data and checks that no other tenant ever receives it. The test is only as good as the marker and the matching rule.
-
Keyed Pseudonymization
A keyed pseudonym is a keyed hash of an identifier. Someone who holds only the pseudonyms, with no access to the key or to a service that computes it, cannot test guesses against them offline, which a plain hash does not prevent. The key then becomes the thing to protect, scope, and rotate.
-
Resource Exhaustion
Recognize capacity exhaustion and enforce limits where AI work actually runs.
-
Output Handling
Treat model output as input that needs destination-specific validation and protection.
-
Software Supply Chain Security
Track and protect the origins and changes of software, dependencies, and model artifacts.
-
Model Extraction
Distinguish approximating model behavior from obtaining its original files or training records.
-
Poisoning and Evasion
Compare attacks on model preparation with manipulation of inputs at use time.
-
-
Trust Boundary
Identify where changes in authority or control require explicit checks.
-
Threat and Vulnerability
Separate a possible harmful event from the weakness that could enable it.
-
Attack Surface
Map the interfaces and paths through which an attacker can influence a system.
-
Security Assets and CIA Triad
Distinguish protected assets from confidentiality, integrity, and availability goals.
-
AI Security
Protect the data, models, tools, and services that make an AI application work.
-
-
Access Revocation
Understand how permission removal must reach sessions, credentials, caches, and shared links.
-
Encryption
Distinguish encryption from anonymity and explain why key access still matters.