A tool call turns a language model’s suggestion into a change outside the model. That change may be a read of one document, an email to a customer or a payment between accounts. The model can select the operation and supply valid arguments without sharing the user’s purpose. Prompt injection makes the gap explicit: an attacker can place instructions in content that a tool returns, then let the agent carry those instructions into a more privileged call.
The defensive boundary therefore belongs outside the model. It should treat every tool call as a proposal, grant only the capability required for the current task and check each attempted effect before execution. A sandbox is one part of that boundary. It cannot compensate for a permanent credential, an unrestricted network or a tool whose legitimate powers include the attacker’s goal.
Issue capability for a task, not an agent
An agent often inherits the user’s broad session because that is convenient. It then carries every available permission through every retrieval and planning step. A better design issues a narrow grant for a bounded task: named operations on named resources, with constrained arguments and a short expiry. A request to summarise a supplier’s message needs read access to that message. It does not need the ability to send mail, enumerate the inbox or read address-book entries.
AgentSandbox gives this principle an architectural form. Its persistent component withholds long-term profile data from an isolated, ephemeral task worker. A data minimiser supplies only task-essential context, while an I/O firewall mediates external interactions. The useful lesson is not that every system needs those exact components. It is that sensitive context and external effects should not accumulate in one long-lived execution environment by default.
Implement the grant as a server-issued capability or scoped credential that the model cannot edit. Bind it to the user, task, tool, resource and permitted operation. Keep secrets inside a broker or execution service rather than exposing them in the prompt. Expire the grant when the task finishes or its deadline passes, and revoke it when policy changes.
Short-lived permissions constrain opportunity; they do not reverse an email already sent or data already copied. Narrow grants also make replanning harder when the next operation depends on an earlier result. Handle that uncertainty with a fresh, explicit grant request rather than a standing permission for whatever the agent might need next. A denied request should end in a safe explanation or a request for authorisation, not a search for a more powerful alternative tool.
Validate the call, then authorise the effect
Typed tool interfaces remove ambiguity at the syntactic boundary. JSON Schema can require a string account identifier, an integer amount and a closed set of operation names. The application can reject unknown fields, malformed values and calls outside its fixed tool registry before doing any work.
Schemas are not intent. A perfectly valid send_money object can name the wrong recipient; an allowed read_file call can target a document outside the task. Even a correct operation can be malicious in its current sequence, such as reading confidential material immediately before an unexpected outbound request. Structure tells the enforcement layer what the model proposed. It says nothing about whether that proposal should happen.
Complete mediation means checking every action against current policy and state, not authorising a plan once and trusting all its later forms. Resolve the authenticated principal, the actual resource and the exact operation outside the model. Validate cross-field rules, resource ownership, amount limits, destination policy and the task’s recent data flow. Recheck mutable state at execution time. If the agent supplied a reason, treat it as evidence to inspect rather than proof.
The tool response crosses the boundary in the other direction. Preserve its provenance as untrusted content, validate its transport shape and limit its size before returning it to the model. Do not assume that converting prose into fields removes an injection: a string field can carry instructions just as easily as a document. Constraining a derived value to a number or enum can reduce what an untrusted item controls, but it can also discard information needed for the task.
Isolate execution, files and egress independently
Generated code needs an operating-system boundary as well as an application policy. Run it with an unprivileged identity in a disposable environment. Mount only the required inputs, keep the base image read-only and provide a separate writable scratch area. Do not expose host directories, shared sockets or unrelated service credentials. Set CPU, memory, process, storage and time limits so a faulty loop cannot consume the whole worker pool.
Filesystem isolation does not imply network isolation. Default to no egress, then route permitted traffic through a proxy that enforces the allowed scheme, destination, method and payload limits. Check redirects and resolved addresses rather than matching only the first URL string. Separate fetching a public reference from contacting an authenticated internal API; they belong behind different grants. Where possible, let the broker perform the narrow operation instead of handing general network access to generated code.
These layers have different failure modes. A strong execution sandbox can still send an authorised email to the wrong person. A strict network allowlist can still leak data to an allowed endpoint. A disposable filesystem cannot undo an external write, and snapshots cannot roll back another organisation’s database. Choose the isolation strength and startup cost for the workload, but judge it by the paths that remain open rather than by the presence of a container or virtual machine.
Reserve approval for consequential uncertainty
Approval is useful when policy cannot settle a consequential action. Reading one public document may proceed automatically. Updating a draft in a scoped workspace may need only deterministic checks. Sending externally, moving money, deleting records, installing packages or changing production access deserves scrutiny proportionate to its consequence and reversibility.
Show the reviewer the actual effect: recipient, amount, resource, content diff, destination and relevant data provenance. Bind approval to the exact validated call, grant and expiry. Any material change requires a new decision, and the approval should authorise one execution rather than a reusable permission. A vague “allow this agent?” prompt merely moves an opaque decision to a busier person.
Risk-based approval also limits fatigue. If every file read produces a dialogue, reviewers learn to accept everything. Batch only genuinely independent, low-risk operations; never collapse several external effects into one ambiguous confirmation. Human review remains fallible, especially when malicious content is hidden or the displayed diff omits a downstream effect. Keep deterministic restrictions in force after approval.
Record the decision before calling it an audit trail
An audit trail connects the task and policy versions to the proposed tool, validated arguments, grant identifier, decision rule, approver, execution result and observed side effects. Include denials, timeouts and retries. An idempotency key can connect a repeated request with one external effect and prevent a transport retry from repeating a payment or write.
Traces are not prevention. Recording an unauthorised payment after execution supplies evidence, not a barrier. Prevention requires the authorisation and execution path to reject the call before the effect. Nor is an application log automatically complete or tamper-resistant. Protect security-relevant records separately where the threat model requires it, and test whether the record captures actual tool outcomes rather than the model’s account of them.
Instrumentation creates its own sensitive store. Arguments, retrieved content and results may contain personal or commercial data. Record structural fields and hashes where they suffice, redact values that reviewers do not need, restrict access and set retention periods. Execution tracing makes agent runs easier to inspect; it does not make every logged claim true.
Evaluate the boundary under attack
A benign test proves that a permission permits useful work. It does not show what that permission lets an attacker achieve. AgentDojo addresses both sides by running agents in stateful environments where tools read and write mutable state. Its deterministic checks inspect output and state changes for user-task utility and attacker success. That distinction should shape production evaluation.
Build cases around injections in content the agent must process, schema-valid calls with unauthorised targets, read-then-send sequences, redirected egress, expired grants and ambiguous approval requests. Include attacks that need the same tool as the user’s task. AgentDojo’s tool-filtering analysis identifies this limitation: removing write tools helps with read-only tasks, but cannot protect a task whose required capability also implements the attacker’s goal.
Measure ordinary task success, utility under attack and attacker success separately. Add false denials, approval frequency, latency and resource use so a defence cannot buy a better security score by stopping every task. Replay known attacks against each policy or model change, then introduce adaptive cases that target the new controls. Keep realistic end-to-end tests because mocks omit retries, partial writes and integration behaviour.
AgentSandbox remains a conceptual framework with preliminary evaluation, and this layered architecture offers no formal security guarantee. Its production value is narrower: fewer reachable effects, clearer decisions and better evidence when a control fails. That is the standard to design and test against.
Sources and further reading
- Zhang et al., “LLM Agents Should Employ Security Principles”, v1 (29 May 2025) — AgentSandbox, least privilege, complete mediation, data minimisation and the limits of current guarantees.
- Debenedetti et al., “AgentDojo: A Dynamic Environment to Evaluate Attacks and Defenses for LLM Agents”, v2 (18 July 2024) — stateful adversarial evaluation, utility and attack metrics, and tool-filtering limitations.
- Beurer-Kellner et al., “Design Patterns for Securing LLM Agents against Prompt Injections”, v3 (27 June 2025) — constrained agent patterns, action sandboxing, strict formatting and confirmation trade-offs.
- Dong et al., “DeltaBox: Scaling Stateful AI Agents with Millisecond-Level Sandbox Checkpoint/Rollback”, v1 (21 May 2026) — coupled filesystem and process rollback, with an explicit limitation on rolling back network I/O and external effects.
- OpenAI, “Function calling and other API updates” (13 June 2023; updated 20 July 2023) — JSON Schema function arguments and warnings about untrusted tool output and consequential actions.
- OpenAI, “New tools for building agents” (11 March 2025) — execution tracing, sensitive-task confirmations and environment-isolation assistance.