An agent reviewing refund requests has read-only access to orders and policy documents, plus a tool that can issue a refund. It can choose what to inspect and when to act. That flexibility helps when a request does not fit a fixed script, but it says nothing about whether the agent will finish correctly. It might keep searching after finding the answer, repeat a failed call, or mistake a confident summary for a completed refund.
Greater autonomy is not reliability. Reliability comes from controlling the run around the model: what counts as done, which failures permit another attempt, how much work remains, and who receives control when the agent cannot proceed safely. The model can propose the next action; application code must own those decisions.
Put a controller around the reasoning loop
ReAct established a useful pattern: interleave reasoning with actions and observations. A thought can decompose a goal or revise a plan after an unexpected result; the action obtains evidence from outside the model. This is a mechanism for adapting a trajectory, not a guarantee that the trajectory will terminate well. The paper also documents repeated thoughts and actions, and unhelpful search results that derail recovery.
A production run needs a controller outside that loop. Its state should include the goal, collected evidence, completed subgoals, attempted actions, known side effects, turn count, retry counts and deadline. It should also define four terminal outcomes: completed, blocked, failed and handed_off. A response without a tool call is merely a model event until the controller maps it to one of those outcomes.
This separation matters because model prose cannot enforce its own limits. An instruction to stop after five turns may be forgotten or misinterpreted. A counter in code cannot be talked out of incrementing. Keep the plan in the run state, but keep the rules for ending the run in deterministic control flow.
Follow one decision cycle
For the refund agent, the initial plan might be to identify the order, retrieve the policy in force at purchase, check eligibility, then either issue the permitted refund or explain the decision. These steps make completion testable. A vague goal such as handling the refund offers no such predicate.
Tool selection should narrow uncertainty with the least authority required. The agent first chooses an order lookup, not the payment tool. The returned order supplies a customer-approved amount and purchase date. It then selects the policy reader using that date rather than relying on the model’s recollection of current terms. Clear tool names, argument schemas and distinct read and write interfaces help the model choose, but the controller still authorises each call against the current state.
The observation changes the plan. Suppose the order lookup shows that the requested refund was already processed. The agent should not issue it again. Nor should it immediately announce success: the controller must compare the existing refund’s amount and state with the request. A matching completed refund satisfies the goal; a pending or partial refund requires a different next step.
Now suppose the policy permits the amount, but exceeds the agent’s automatic-payment limit. Replanning cannot enlarge permission. The right transition is handed_off, with the verified order, policy reference and proposed action attached. Replanning means choosing another permitted route towards the same goal, not rewriting the goal or escaping a constraint.
Completion needs evidence
OpenAI’s guide describes exits on a final structured output, a response without tool calls, errors or a maximum turn count. Those are loop mechanics. An application must add the semantic check that distinguishes a finished task from a finished utterance.
For this workflow, the completion predicate might require a refund record in a terminal successful state, an amount equal to the authorised request, the expected order identifier and no duplicate mutation from this run. A read-back from the payment system supplies stronger evidence than the agent’s assertion. Where the request asks only for an eligibility decision, the predicate instead checks the required policy evidence and decision fields.
Negative results also need explicit completion conditions. ineligible, not_found and needs_customer_input can be valid outcomes when supported by evidence. Forcing every run into a success shape encourages unnecessary searches or invented justifications. If the available systems cannot verify the predicate, the controller should report that limitation rather than lower the standard until the agent passes.
Spend retries only on new information
A turn budget bounds the whole search; retry budgets bound repeated attempts at a particular operation. A deadline handles slow dependencies that would otherwise consume turns without producing observations. These counters answer different questions and should not be hidden inside one large maximum.
Retry only when another attempt can change the result. A transient service failure may justify bounded backoff. Malformed arguments need correction before resubmission. An authorisation denial needs a different principal or human decision, not another identical call. Repeating the same query and receiving the same observation is evidence of no progress, even if no exception occurred.
Reflexion makes this distinction concrete. Its trials use task feedback to produce verbal reflections for later attempts, while bounding stored experiences and trials. In its ALFWorld setup, repeated action-response cycles and excessive action counts trigger reflection or reset. Its limitations are equally instructive: reflection can fail to supply a useful new strategy, and some tasks show no meaningful improvement after several trials. Reflection is worth a budget only when it changes the next decision.
Budget values are workload-specific. A tight cap can stop a valid long task; a loose cap buys more opportunities for recovery but increases latency, cost and exposure to unwanted actions. Measure the distribution of successful turns and the value of later progress rather than copying a universal limit. Toolformer offered an early, stark example of this trade-off: its evaluation allowed at most one API call per input to prevent endless calls, which also ruled out useful chained tool use. A production controller should permit necessary sequences while making each additional step earn its place.
Make errors choose the next state
An error is not a generic invitation to try again. It is a typed observation that should select a branch.
- Invalid request or policy rejection: stop or return a supported explanation. Rephrasing the same prohibited action is not recovery.
- Authentication or permission failure: halt the affected path and request the missing authority. Do not search for a more powerful tool.
- Transient read failure: retry within a small attempt budget and the run deadline, using backoff. Preserve the original query and correlation identifier.
- Uncertain write result: do not assume failure. A timeout may occur after the payment system commits. Reconcile with a read before any repeated write, or retry only with the same idempotency key where the payment system supports it; if the outcome remains unknown, hand off.
- Malformed or untrusted observation: reject it as evidence and fail safely when no trustworthy alternative exists.
An exit must leave the environment in a defined state. Cancel outstanding work where possible, prevent further writes and record any effects already observed. failed should mean the task did not complete, not that its fate is unknown. That distinction is operationally important when a person must reconcile a payment later.
Design the handoff before deployment
Human handoff is not an error message with a ticket number. It is a controlled transfer of responsibility. Trigger it for sensitive or irreversible actions, exhausted recovery budgets, conflicting evidence and goals that need judgement beyond the agent’s authority. The OpenAI guide specifically recommends failure thresholds and oversight for high-risk actions; both belong in the run definition rather than an emergency add-on.
A useful handoff contains the original goal, verified facts, unresolved question, proposed next action, attempts made and observed side effects. Include policy and tool versions so the reviewer knows which rules governed the decision. Do not make the reviewer reconstruct the trajectory from raw model prose.
Pause execution while waiting. When the reviewer responds, resume from a validated checkpoint or start a new run with fresh authorisation and budget. A resumed model should receive the reviewer’s decision as evidence, not inherit an expired assumption about account state.
Test the exits, not just the answers
Evaluate successful completion, premature completion, no-progress detection, budget exhaustion, safe failure and handoff quality as separate behaviours. Inject stale records, repeated observations, timeouts before and after writes, contradictory policies and cases that need approval. Assert final state and side effects alongside the visible response.
AgentBoard shows why final success alone hides useful detail: its progress measure tracks states reached against goals or annotated subgoals. Its reliance on human annotation also shows the cost of defining meaningful intermediate checks. Production teams face the same work. They must specify which states represent progress and which merely look busy.
A bounded agent may attempt fewer tasks than an unrestricted loop and still be the better system. It finishes when evidence supports completion, stops when further action cannot help, and transfers control with enough state for a person to continue. Those exits are not limits on the design’s ambition; they are what makes its autonomy operable.
Sources and further reading
- OpenAI, “A practical guide to building agents” — single-agent run exits, retry limits, failure escalation and high-risk human oversight, PDF edition created 7 April 2025; guide listing dated 15 January 2026.
- Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models”, v3 — interleaved plans, actions, observations, exception handling and repetition failures, 10 March 2023.
- Shinn et al., “Reflexion: Language Agents with Verbal Reinforcement Learning”, v4 — bounded trials, feedback-driven reflection, repeated-cycle detection and recovery limitations, 10 October 2023.
- Schick et al., “Toolformer: Language Models Can Teach Themselves to Use Tools”, v1 — tool selection and the one-call cap used to prevent loops, with chained-use limitations, 9 February 2023.
- Zhang et al., “AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents”, v2 — trajectories, final success, subgoal progress and annotation limitations, 23 December 2024.