AI Agent Tool Execution and Reliability

Consider a fictional retailer whose support agent could look up orders, create return labels, issue refunds up to the order value, and email customers. It handled two-thirds of return requests without a person, and for its first month the team was pleased. Then the finance reconciliation found four kinds of problem. A customer had been refunded twice: the payment service had timed out, the agent had read the error and called the refund tool again, and the first call had in fact succeeded. Another customer, asking to cancel a monthly plan, had been refunded the annual amount; the refund tool accepted any amount up to the order total, and the model had chosen the wrong one. A server restart in the middle of several conversations had started them over, and those customers received two return labels each. And one conversation had called the carrier-tracking tool forty times in six minutes after an ambiguous error, until the carrier’s rate limit blocked every other conversation too.

These incidents combine proposal errors with missing execution safeguards. Choosing the wrong refund amount is a model or task-interpretation error; allowing it to pass is also a failure of the payment boundary. Better proposals can reduce mistakes, but the application still has to enforce permissions, financial limits, duplicate protection, and recovery rules. An agent lets a model help choose subsequent steps, while the runtime remains responsible for what those steps are allowed to do.

The model proposes; the application executes

An AI agent runs a loop: the model reads the task and what has happened so far, chooses an action, the application carries it out, and the result goes back to the model for the next choice. The common form of this loop is the ReAct pattern of thought, action, and observation. Each action is a tool call, a structured request naming a function and its arguments. Google’s function-calling documentation is explicit about the division of labor: “The model doesn’t execute the function itself.” The application does, and its guidance lists as a best practice: “Validate function calls before executing.”

That division gives the application four places to enforce rules on every action: when the call is proposed, when it is validated against scope and policy, when it is executed, and when its result is recorded. Validation and execution guards can prevent unsafe effects; recording supports recovery and investigation but cannot retroactively prevent an effect that has already happened. None could be stopped reliably by instructions in the prompt, because a prompt shapes what the model is likely to propose; it does not constrain what the application will do.

Before giving a model control of the sequence at all, it is worth asking whether the task needs it. Google’s guidance on agent design patterns separates deterministic workflows, where the steps are known in advance and the application fixes their order, from patterns where the model directs the process. A return request that always means look up order, check eligibility, create label, refund, email is a workflow with a model filling in details, and it is easier to test and reason about than an open-ended agent. Workflow orchestration covers the machinery for fixed sequences.

Scope: what the agent can touch

Each tool is a grant of power, and the set of tools defines the blast radius of any mistake. The principle is least privilege, applied to both operations and resources: reading one customer’s orders is a different permission from reading all orders, and reading anything is a different permission from issuing a refund. Four design choices follow.

• Separate reading from acting. Read tools also need purpose, resource, tenant, volume, and cost limits. Reading private data can expose it even without changing the source. State-changing tools need additional safeguards for their business effects.
• Enforce limits in the tool. Validate the requested cancellation scope, billing period, currency, and refund items; clarify ambiguous intent before computing an amount. Compute the remaining refundable balance after prior refunds and pending reservations. Enforce that balance atomically, for example through a reservation or conditional update, so two requests cannot both spend the same remaining amount. Per-operation deduplication and an aggregate refund limit are different safeguards. A limit stated only in the prompt is a suggestion.
• Act as the user where possible. When the agent works on a customer’s behalf, use a supported delegation path and enforce effective permissions for the specific order and tenant in the downstream service. A user identity alone does not prove that the service checks ownership correctly or that the user has access to only one customer’s resources.
• Contain code execution. An agent that runs code needs a sandbox with limits on files, network, and resources, remembering that isolation limits damage without making the code correct.

Approval: which actions wait for a person

Some actions should not happen until a person agrees. Google’s guidance describes a human-in-the-loop pattern in which, at a predefined checkpoint, “the agent pauses its execution” and waits for a person to approve, correct, or supply input. The question is which actions deserve it. Consider consequence, reversibility, user intent, delegated authority, and the applicable policy. A return label is cheap and can be voided; a pending refund might be cancellable under the provider’s contract, while recovering a completed refund can require a separately authorized process; a new charge is not automatically permitted; a bank transfer or a deleted record may not be reversible at all. Refunds above a set value, any action outside the normal pattern, and anything irreversible are natural candidates.

An approval is only as good as what the approver sees. A prompt that says only Approve refund? invites a reflexive click, which is why human oversight treats a bare approval as weak evidence of supervision. The request should show the exact action and arguments that will execute, the customer and order, the basis for the amount, and what the agent observed to reach it. Bind approval to the business operation ID, approver authority, target and arguments, relevant policy or resource versions, and an expiry. Deduplicate approval events. Before execution, recheck permission and business preconditions: unchanged arguments can become invalid if another refund has reduced the balance. Changed scope or stale approval must follow the policy for rejection, cancellation, or renewed approval. Operations within a delegated automatic allowance should record that policy decision rather than fabricate a human approval.

Retries without duplicates

The double refund came from the most common failure in distributed systems: a request whose outcome is unknown. When a call times out, the operation may have failed, may have succeeded, or may still be running. An unprotected retry can create another effect if the first attempt succeeded or later commits. A retry protected by a valid idempotency contract can instead recover the same operation’s result without assuming failure. The model cannot resolve this by reasoning, because the information it needs is not in its context.

The recovery contract belongs to the tool layer. For a non-idempotent business action, create and persist a stable operation ID before sending the request. Keep it distinct from conversation turns, model tool-call IDs, and attempt counters. The same refund intention must reuse its operation ID and arguments across retries and restarts, even if the model proposes it in a later turn. A genuinely new partial refund needs its own identity.

An idempotency key implements this only when the receiving service honors its scope, parameter binding, concurrent-request handling, and retention period; merely sending a key does not make an operation idempotent. Naturally idempotent resource operations may use a different guard. Stop automatic retries before protection expires and retain a recovery path for unresolved work.

Return to the timed-out refund: the service can complete the payment before the caller receives or records the result. The next step is therefore to recover the status of that persisted business operation, not to treat the timeout as a fresh refund instruction. Track completed, in-progress, confirmed-not-applied, and unknown outcomes separately. A status lookup that returns no record does not prove the original request cannot commit later. Without duplicate protection, retry only when authoritative evidence establishes that no effect occurred and the earlier attempt can no longer apply it, or when the dependency provides another safe recovery protocol. Otherwise continue reconciliation or hand the operation to an owner without issuing a new unprotected effect.

Give one layer responsibility for retries, using bounded attempts and elapsed time, exponential backoff, jitter, and the documented Retry-After delay. Report pending or unknown status to the model without letting a new proposal bypass that recovery state.

Classify errors using the operation contract and timing, not the status label alone. A newly created tracking number can be temporarily absent while the carrier propagates it; a malformed number may never appear. A rate limit or quota problem may persist despite waiting. Permit only policy-approved, safe retries within a deadline, and suppress uncontrolled repetition by the model. An identical retry is not inherently wrong when it is exactly what the recovery protocol requires.

State that survives a restart

The duplicate labels came from keeping the agent’s progress in the memory of one process. An agent’s state is more than its conversation: it includes which steps have completed, with the identifiers the external systems returned, which are pending, and which await approval. If that record lives only in memory, a restart loses it, and the application must reconstruct progress from trustworthy external operation records or stop and reconcile uncertain effects. Blindly starting over is not a safe default.

Persist the operation identity and execution intent before the external call, then record pending state and the eventual outcome durably. These writes do not by themselves commit atomically with a remote payment. Durable execution frameworks do this systematically: they checkpoint progress and, after a restart, replay the task using results for steps whose completion is durably recorded. An effect can have happened even when that completion record is missing. They also state the limit plainly. Microsoft’s Durable Functions documentation says “the runtime guarantees that each activity runs at least once during an orchestration”, meaning a step that finished just before a crash may run again, and it recommends making such steps idempotent. Durable state and idempotent tools work together; neither is enough alone.

Durable state also supports long approval waits without holding a process open, but the approval and current execution preconditions still need revalidation. Non-deterministic model calls belong behind replay-safe activity boundaries, with their results and chosen actions recorded for reuse; changes to running workflow code or tool contracts need compatible versioning or migration.

When a later step fails permanently after earlier ones have taken effect, the earlier effects do not disappear. A label created and a refund issued before the email service fails are still real. Undoing them is new work, a compensating transaction such as voiding the label, and a multi-step task that may need to unwind is designed as a saga, with the recovery policy defined for each step. Some effects have no permitted compensation. An email failure after a valid refund may call for retrying delivery or manual notification rather than reversing the refund. Identify the boundary after which the workflow commits to forward completion, and assign an owner for permanent or compensation failures.

Knowing when to stop

A model-driven loop continues until something ends it. Google’s guidance names the danger for looped agent designs directly, “the risk of an infinite loop”, with the cost and resource consumption that follows. Stopping rules are enforced by the application: a maximum number of steps, a time limit, a token or cost budget per task, and a limit on consecutive failures. The agent should also have explicit ways to end other than success: reporting that it lacks information, that an error cannot be recovered, or that a person must decide. Define which endings require an owner and an acknowledgement. Reaching a limit stops new calls; it does not prove that in-flight work was canceled. Track pending external effects through reconciliation or hand-off, using the dependency’s cancellation contract rather than treating a local timeout as rollback.

Limits protect shared systems as well as budgets. The forty-call loop hit the carrier’s rate limit for the whole company. Per-conversation limits constrain one task but do not bound the combined load. Add service- and tenant-level rate, concurrency, and retry budgets; count tool retries, status polling, and child calls as well as model-visible steps. Twenty tasks making ten calls each still exceed a shared hundred-call allowance. The loop mechanics, including a step limit in code, are shown in Retrieval-Augmented Generation and Tool-Using Agents.

Connecting to external systems

A tool is an interface to someone else’s system, and it deserves the care of any integration. Its definition, the name, description, and parameter schema the model sees, should describe precisely what it does and when to use it, since the model chooses tools by reading those descriptions. Its arguments should be validated as structured output is: a well-formed call can still name the wrong order. Its errors should distinguish not found, not permitted, invalid input, temporarily unavailable, and outcome unknown, because each calls for a different next step. Scope credentials to the required privileges, audience, and trust boundary. Separate them when those boundaries differ; distinct tool names do not by themselves require distinct credentials, and shared credentials must not lend broader authority to a narrower operation. Protocols that standardize how tools are described and discovered make connecting systems easier; they do not supply any of these properties.

Every proposed call, validation decision, approval, execution, and result should be recorded with the task and user it belongs to. An audit log that separates what was requested from what actually happened is what let the retailer’s finance team reconstruct the double refund, and it is what an investigation of any agent action will need.

What a reviewer should ask

• Does this task need the model to choose the sequence, or would a fixed workflow do?
• Which tools change state, and are their limits enforced inside the tool?
• Whose identity and permissions do downstream calls use?
• Which actions require approval, and does the approver see the exact call?
• What happens when a state-changing call times out?
• Where is task progress stored, and what happens on restart?
• What limits stop a task, and who receives it when it stops?
• Can every executed action be traced to its task, principal, operation ID, and applicable approval or delegated-policy decision?

Three things are offered as evidence that an agent is safe to run and are not. Instructions in the prompt telling the model not to repeat refunds or exceed amounts, which shape proposals but enforce nothing. A demonstration of successful end-to-end tasks, which shows the happy path and none of the timeouts, restarts, or ambiguous errors that production supplies. And an approval step, without a check of what approvers actually see and how often they decline.

The revised design for this fictional retailer persists an operation ID before each refund or label request and reuses it for the same business action within the provider’s recovery contract. An unknown outcome stays pending until authoritative evidence or reconciliation resolves it; an empty status lookup alone does not permit a new effect. The refund path verifies cancellation scope and enforces the remaining balance under concurrency. Required approvals are bound to the operation and rechecked before execution. Durable history reconstructs recorded progress, while duplicate protection handles activities that run again after an effect but before its result was saved. Task limits and shared service budgets constrain new work, and unresolved effects have an owner. Test lost responses, crashes on both sides of result recording, expired keys, stale approvals, concurrent refunds, and delayed status visibility before claiming that these controls prevent duplicate payments.

References:

• Google AI for Developers, Gemini API: Function calling (Creative Commons Attribution 4.0).
• Google Cloud Architecture Center, Choose a design pattern for your agentic AI system (Creative Commons Attribution 4.0).
• Microsoft Learn, Durable Task programming model (Creative Commons Attribution 4.0).