Log every action before an agent retries
When an agent times out after taking an action, retrying can duplicate the work. Give every attempted action a durable record, a status and a recovery rule before trusting automation.
On this page 5 sections
- Treat a timeout as an unknown outcome, not proof that an action failed.
- Record intent, execution state and the external result for each consequential action.
- Make retries depend on the action's idempotency and recovery path.
An agent calls a tool to create a booking. The request times out. Did the booking fail, or did the tool create it and lose the response? If the agent simply tries again, the user may get two bookings. If it gives up, the user may be left with none. A reliable agent needs a record of what it tried to do and a rule for resolving the unknown state.
A timeout is not an outcome#
A tool call has at least two separate events: the system receives a request, and the agent receives a response. A timeout tells us the second event did not happen in time. It does not tell us whether the first event caused a change.
This distinction matters whenever a tool can create an external side effect: sending a message, issuing a refund, submitting a form, changing access, or reserving a resource. The agent’s conversational context may say “I tried,” but that is not evidence about the external system.
Start by classifying each action in a workflow:
| Action type | Example | Safe default after timeout |
|---|---|---|
| Read-only | Fetch a status | Retry within a bounded limit |
| Naturally idempotent | Set a record’s status to a specific value | Retry if the operation truly replaces the same state |
| Non-idempotent | Create a new booking | Check for the original result before repeating |
| Irreversible or high impact | Send a payment instruction | Stop and reconcile or request human review |
“Idempotent” needs to describe the actual tool behavior, not the intention in the prompt. Setting a field to “approved” may be repeatable; calling “approve next item” may approve a different item on the second attempt. Ask what the external system does when it receives the same request twice.
Give each action a durable identity#
For every consequential action, create an action record before calling the tool. It should have a stable identifier that survives model retries, plus enough context to reconcile the call without storing unnecessary sensitive data.
A minimal ledger entry could contain:
- Action ID: a unique identifier for this intended operation.
- Intent: the specific change requested, in a compact and inspectable form.
- Target: the external record or resource, if known.
- Status: prepared, submitted, confirmed, failed, or needs reconciliation.
- Attempt details: timestamps, tool response identifiers, and error category.
- Recovery rule: what the system may safely do next.
The key is to persist the action ID before sending the request. If the process crashes after the external action but before updating its own state, the record still tells the recovery path that an operation may be in flight.
Where the external service supports an idempotency key, pass the same action ID on each retry of the same intended operation. Do not generate a fresh key just because the previous response was missing; that would turn a retry into a new operation. If the service has no idempotency support, use a lookup or reconciliation step before attempting another write.
A ledger is not a transcript of the agent’s reasoning. It is a state machine for side effects. Keep it small enough that software can enforce it and an operator can understand it during an incident.
Separate intent from execution#
A useful state model distinguishes what the agent decided from what the external system confirmed. One practical sequence is:
- Prepared: the action and its authorization are recorded, but no request has been sent.
- Submitted: the request was sent; the outcome may still be unknown.
- Confirmed: a response or subsequent lookup proves the intended result exists.
- Failed: the system has evidence the action did not happen and will not complete later.
- Needs reconciliation: the outcome is ambiguous or the system cannot safely decide.
Avoid treating a caught exception as equivalent to “failed.” A connection reset after submission is ambiguous; a validation error returned before execution may be a definite failure. The tool adapter should classify errors according to its contract, not ask the language model to guess from an arbitrary message.
The agent’s user-facing language should follow the same distinction. If the action is merely submitted, say it is being checked. Do not claim success until there is confirmation. If the result is unknown, explain that clearly and avoid inviting the user to repeat the request before reconciliation.
Make retries a policy, not a reflex#
Retry logic should be attached to the action type and error category. A generic “retry three times” wrapper can multiply side effects while appearing robust.
A decision rule can be simple:
- If the operation is read-only and the error is transient, retry with a bounded delay.
- If the operation is idempotent and uses the same key, retry that same operation.
- If the operation is non-idempotent and the outcome is unknown, query by action ID or target before any new write.
- If reconciliation cannot establish the outcome, mark it for a person rather than guessing.
- If the error is a permission or validation failure, do not retry unchanged; surface the missing prerequisite or correct the input.
Set a maximum retry window as well as a count. A request that remains ambiguous after the useful recovery period should leave the automated path and enter a visible queue. Otherwise, retries can continue long after the user has moved on, or a backlog can quietly accumulate.
For example, suppose an agent submits a support ticket through a tool that has no idempotency key. The request times out. The recovery worker searches for a ticket matching the action’s external reference. If exactly one match exists, it marks the action confirmed. If none exists and the service can guarantee the earlier request is no longer processing, it may submit once more. If there are multiple matches or the search is inconclusive, it routes the record for review. The rule is explicit about what the system knows and what it does not.
Design reconciliation before launch#
A ledger only helps if the system can use it to resolve uncertainty. During design, answer these questions for every external write:
- Can the destination accept an idempotency key or client-generated reference?
- Can the system query for the result using that reference?
- How long can a request remain in progress after the caller times out?
- Who owns unresolved actions, and where do they see them?
- What should the user see while the result is unknown?
- How can an operator safely retry, cancel, or mark the action resolved?
Test the failure sequence, not only the happy path. Simulate a timeout before the request reaches the service, after the service commits but before its response returns, and while the destination is still processing. The expected result should be different where the evidence differs: no action to reconcile, a completed action to recognize, or an in-flight action to wait on.
Track operational signals that reveal whether the design is working: actions in the needs-reconciliation state, their age, duplicate effects found, and manual resolutions. These are not interchangeable with model-quality scores. They tell you whether the product can account for its side effects.
Before an agent can perform a consequential write, complete this checklist: stable action ID; persisted intent; known idempotency behavior; timeout classification; reconciliation query; bounded retry rule; owner for unresolved outcomes. If any item is missing, keep the action behind a human confirmation step until the recovery path exists.