Confused Deputy
A confused deputy is a program that holds authority of its own, receives a request from a party that lacks that authority, and uses its own authority to carry the request out. The program is not compromised. It does exactly what it was built to do, but it applies the wrong authority to the request, so a caller reaches something the caller could not reach directly. Norm Hardy gave the problem this name in a 1988 paper about a real incident on a timesharing system.
The original case
A compiler on the Tymshare system was allowed to write to a directory that also held the billing file. A user could pass the compiler the name of a file to receive debugging output. A user passed the name of the billing file. The operating system saw a program with permission to write there and allowed the write, and the billing data was overwritten. The user had no right to modify the billing file; the compiler did, and the system could not tell which of the two rights the write was meant to use. The compiler combined two authorities, its own and its caller’s, and the check used the wrong one.
The same shape in AI tool services
An AI tool service is a deputy in this sense. It runs with a credential, often broad so that one service can handle every tenant, and it takes instructions that originate in a model, which may be following text written by a user or by a stranger whose document was retrieved. If the service checks the request against its own credential, the model can pass any path or identifier the credential can reach. The demonstration below reads a file through two versions of a tool: one that presents the service’s own scope, and one that presents the scope of the calling tenant. The request comes from tenant acme and names a file that belongs to birch, as injected text might ask.
FILES = {"acme/report.txt": "acme quarterly report", "birch/report.txt": "birch quarterly report"}
def storage_read(credential_scope, path): # the resource checks only the presented authority
if not path.startswith(credential_scope): raise PermissionError(path)
return FILES[path]
SERVICE_SCOPE = "" # tool service credential: all tenants
def read_tool_deputy(caller_tenant, path): # confused: uses its own authority
return storage_read(SERVICE_SCOPE, path)
def read_tool_scoped(caller_tenant, path): # acts with the caller's authority
return storage_read(caller_tenant + "/", path)
request = ("acme", "birch/report.txt")
for tool in (read_tool_deputy, read_tool_scoped):
try: print(tool.__name__, "->", tool(*request))
except PermissionError as e: print(tool.__name__, "-> denied:", e)
read_tool_deputy -> birch quarterly report
read_tool_scoped -> denied: birch/report.txt
The storage layer is identical in both runs. Only the authority attached to the call differs. In the first, the question the system answers is whether the service may read the file, and it may. In the second, it is whether the caller may, and the caller may not. The example isolates this authorization decision.
Preventing it
- Carry the caller’s authority with the request. Use a credential scoped to the calling tenant or user, or a short-lived token minted for that request. OWASP’s guidance on excessive agency recommends executing extensions in the user’s context so that downstream systems see that user’s scope, with the minimum privileges necessary.
- If the deputy must hold broad authority, make it check the caller. Before using its own credential, it verifies that the authenticated caller may access the named object. This works only if every tool remembers to do it, which is why scoped credentials are the more robust choice.
- Do not let the request name its own identity. The tenant and user come from the authenticated context, not from arguments the model fills in.
- Reduce what the deputy can reach. Narrow tools with narrow credentials limit the damage when one is misled.
The related idea of least privilege shrinks the deputy’s authority; confused deputy prevention ensures the authority used is the caller’s. Both are needed, and neither depends on the model resisting a manipulated prompt.
References
Discover more from Insightful Data Lab
Subscribe to get the latest posts sent to your email.
