AI Agent Retries: Where to Resume Safely

An agent saves a document, starts the next part of the task, and loses its connection. Retrying the whole task sounds helpful. It can also repeat the operation that already succeeded.

My rule is to recover the smallest unit whose effects we can account for. Sometimes that is an unfinished model request. Sometimes it is a tool operation with a stable identity. Sometimes the only correct next step is to check what happened. A retry delay cannot make that decision for you.

This is becoming a practical question as agent platforms take responsibility for execution. Salesforce introduced its Enterprise AI Harness architecture on September 11, including shared action and governance controls. The READY research preprint, submitted September 2, separates benchmark performance from qualification for a particular deployment. Neither establishes that an arbitrary failed agent can safely start again. Recovery needs its own contract and its own tests.

Why can an HTTP 200 response still leave an agent unfinished?

A successful HTTP status tells you that the streaming response began successfully. It does not guarantee that the application received a complete result. A connection can close after an initial event, or a stream can end without the completion event the runner needs.

This creates an awkward failure mode: the transport finishes normally, but the agent has nothing complete to consume. Code that only catches network exceptions misses it. The absence of an exception is weaker evidence than the presence of the expected terminal result.

Define completion at the protocol layer. Track whether the required successful terminal event arrived. If the stream ends first, report an incomplete generation. An explicit unsuccessful terminal event is a different case and should retain its meaning. Do not turn every unsuccessful ending into a transient connection failure merely because retrying is convenient.

Should you retry a model request or the whole agent?

Those are different operations. A whole agent turn may include completed tool work and several model requests. Restarting it from the original instruction can put completed work back on the execution path. Repeating the current model request can preserve earlier tool results as input.

That narrower boundary only works if you know the runner's dispatch contract. For the native function tools exercised in our replay, the runner executes a proposed call after a complete model response, not while its arguments are still arriving. An unfinished proposal therefore differs from an executed operation. Another runtime, or a tool executed by an upstream service, may behave differently.

Our earlier article on streaming an AI agent discussed a conservative whole-turn retry guard. The question here is narrower: can recovery stay inside an unfinished generation even after an earlier generation has produced useful tool work?

What did the Norman recovery replay demonstrate?

We exercised Norman's assistant wrapper with the actual agent runner and synthetic tools. The fixture saves a fictional receipt, then links it. These operations append markers to a local list; they do not write bookkeeping records. We interrupted model generation both before that work began and after the saved-result marker already existed.

The older wrapper did not detect a normally ended stream that lacked completion. In the updated wrapper, an eligible incomplete generation receives a bounded retry with the same input. In the later interruption case, that input still contains the earlier tool result. The fixture reaches its final answer with the intended save-and-link sequence and no repeated earlier marker.

We also passed a synthetic HTTP 200 event stream through the real HTTP/SSE decoder and runner. A missing completion event was detected, and the next eligible attempt recovered. This check matters because a fake iterator alone cannot establish what the decoding layer delivers to the wrapper.

This is a regression demonstration against Norman code, not a production reliability benchmark. We made no live model calls and queried no customer database. The useful result is a verified boundary, not a percentage to put on a landing page.

Which events should stop an automatic retry?

An empty stream and a half-displayed answer are not interchangeable. Starting the answer again can duplicate text, confuse downstream consumers, or contradict content the user already saw. Completion itself also closes the restart window: a later transport error must not turn a finished response into permission to regenerate it.

Work performed inside an upstream service is another boundary. The application may not have the same visibility into its effects as it has into native tools. In the inspected wrapper, hosted-tool activity and unrecognized event types are handled conservatively rather than presumed harmless.

Cancellation must keep its meaning too. A user stopping work is not asking the retry mechanism to continue it. The local checks cover cancellation and a request deadline interrupting retry backoff. A retry budget belongs inside the original execution budget; it should not quietly create a fresh lifetime for the task.

Does this make the agent durably resumable?

No. Keeping an earlier tool result inside a running process does not prove that the workflow survives a worker crash. Durable recovery needs persisted progress and a way to reconnect that progress to completed operations. Our agent harness article explains the broader execution context; this replay tests only one part of it.

The Apache Airflow agent documentation makes a useful distinction: cached completed results can be reused, but a tool that changes external state before its result is cached can repeat that effect on retry.

For such an operation, “no saved result” is an observation about your records, not proof that nothing happened. Use an operation identity, a destination that supports deduplication, or an authoritative outcome check where available. If the outcome cannot be established, preserve that uncertainty. A local stream-retry fix is not an exactly-once guarantee across external systems.

What should a retry report record?

A report that says only “retry succeeded” hides the important decision. Record which unit was repeated, why it was eligible, which earlier results were retained, and how the resulting state was checked. Separate successful completion from a stopped attempt or an unresolved external outcome.

Here is an illustrative reporting shape, not Norman's API or an executable configuration:

{
  "retry_scope": "current_generation",
  "completion_event": "missing",
  "earlier_tool_results": "retained",
  "restart_boundary": "still_open",
  "verification": "synthetic_effect_sequence_checked"
}

The report should also disclose its limits. Did it exercise an actual decoder, the actual runner, persistence, or a simulated tool? Were sockets disabled? Which revision was tested? Our discussion of agent memory evidence applies here too: a recorded event supports a particular claim, not every desirable conclusion about the system.

How do you test AI agent retries without hiding failures?

Place failures around the boundary you claim to protect. End a stream before completion, interrupt a later generation after a prior tool result, and check the exact synthetic effect sequence. Then exercise refusal cases: visible text, completed output, upstream tool activity, explicit unsuccessful termination and cancellation.

Keep a failure that outlasts the retry budget. Otherwise a test suite can reward a mechanism that always tries again and never reports that it is stuck. Compare against the earlier implementation to show that the regression scenario actually distinguishes the change.

Finally, name the untested failure domains. Our replay does not cover worker loss, external write ambiguity or every client renderer. Those need different fixtures. I would rather ship a recovery rule with a precise boundary than describe every interrupted agent as recoverable. The boundary tells the next engineer where more evidence is needed.