AI Agent Tool Results: Empty Is a Claim

An agent checks a collection and reports: “Nothing is missing.” That sounds modest. It is actually a strong claim: the relevant records were read, the check covered the intended scope, and the evidence supports an absence.

I want that claim to be earned before a model writes the sentence. An empty list, an interrupted read and an unavailable source cannot share the same meaning. Even reading every page is weaker than reading one consistent snapshot.

This matters as agents move into review work. FloQast's September 16 announcement includes AI journal-entry review and transaction anomaly detection. Workiva introduced Agent Studio and automated audit testing on September 15. These are vendor-described capabilities, not independent quality measurements. They make the underlying question urgent: what evidence permits an agent to report that a check found nothing?

Why does an empty tool result need evidence?

“Found no problems” and “could not look for problems” can both produce an empty array in a careless integration. A timeout handler returns a default value; a summarizer counts it; the assistant writes a confident sentence. The final arithmetic can be correct while the conclusion is unsupported.

The error happens before language generation. Once an unavailable source has become an ordinary empty collection, the model no longer has the information needed to distinguish failure from absence. Asking it to be cautious cannot restore a distinction the adapter discarded.

A negative result therefore needs a retrieval contract. State the requested scope, whether its collection was read successfully, and what consistency the source provides. Then describe the finding within those limits. “No missing links among the returned records” is narrower than “all documents are present.” Neither establishes that every real-world document exists in the source.

How do complete, partial and unavailable results differ?

Keep the condition of the read separate from the findings. A complete read may contain findings or none. A partial read may already contain useful observations, but cannot establish the total for the collection. An unavailable read cannot supply a verified count.

A tool may choose not to expose partial observations. Norman's reader in this replay returns an unavailable result instead of a partial summary. That is a deliberate output boundary: downstream code receives no authoritative total from an incomplete collection. A different design could return partial findings, provided their incomplete coverage remains explicit.

What did Norman's tool-result replay show?

We ran a local replay of Norman's document-check reader and summary function using synthetic paginated responses. The real application functions performed the validation and aggregation. The API transport was replaced with a mock; configuration and context were isolated. No customer database, model or live service participated.

An accepted empty collection produced a zero count. When a later page timed out, the function returned an unavailable status without summary totals. Malformed pages, repeated record identities and a changed advertised collection size were also rejected. These checks establish specific behavior for the exercised inputs, not universal data correctness.

We then introduced a deliberately lossy negative control in an isolated copy: replace an unavailable result with an empty list immediately before aggregation. The same failure scenarios now produced ordinary empty summaries, and their assertions failed. This was an intentional mutation for the experiment, not a claim that a previous deployed version behaved this way.

The lesson is about information preservation. Correct handling at the reader is insufficient if the next adapter substitutes a default. Our retry article asks which operation may safely repeat. Here the question is what a completed tool call allows its consumer to conclude.

Why doesn't complete pagination prove a snapshot?

There is a second boundary that error handling cannot solve. Suppose a fictional collection contains records A and B. Before a change, A has a document link and B lacks one. After the change, A lacks the link and B has one. Both source states contain a missing link.

Now read A from the earlier state and B from the later state. The combined response contains both identities, the advertised size stays the same, and neither returned record lacks a link. The summary reports zero. That result describes the assembled responses, but neither of the two source states.

We constructed exactly those page responses and passed them through the same Norman functions. The reader accepted them. This demonstrates a limit of its count-and-identity checks. It is a synthetic counterexample, not an observed production incident or a measurement of how often concurrent changes happen.

Stable ordering helps a reader traverse a collection; it does not freeze changing values. A stronger guarantee requires a source mechanism such as a snapshot-bound cursor or a versioned export. Where that guarantee is unavailable, describe the observation as a read over an interval. Do not quietly promote it into a statement about one instant.

How should an agent tool report an unavailable result?

Use a result shape that makes absence and uncertainty different states. The following is an illustrative application contract, not Norman's stored schema or an MCP wire response:

{
  "status": "unavailable",
  "coverage": "incomplete",
  "finding_count": null,
  "consistency": "not_established",
  "next_action": "repeat_the_read"
}

The useful property is not the field names. A missing measurement cannot accidentally become an ordinary zero. Define which states allow counts, require consumers to handle the other states, and test that rule through the rendering boundary. A schema that permits contradictory combinations still needs semantic validation.

The MCP tools specification distinguishes protocol errors from execution errors reported with isError. That protocol distinction and an application's coverage contract answer different questions. An execution error must be communicated as such; a successful call still needs enough context to interpret its data. Our local replay exercised the application result, not MCP serialization or a client's treatment of that flag.

What should the agent say when a check is incomplete?

The wording should follow the evidence. After a failed read: “I could not complete the check.” After a partial read: identify the observations as partial and leave the overall conclusion open. After an accepted read: state what was checked, within which scope, and whether a consistent snapshot was established.

This is especially important when results from several sources feed one answer. A working source must not mask an unavailable one. Preserve their individual outcomes before deciding whether the combined conclusion is possible. A polished paragraph should not flatten a mixed evidence state into a single reassuring status.

We have made a similar distinction for agent memory: recorded activity does not automatically prove a useful outcome. Here, retrieved records do not automatically prove completeness. The agent's job includes communicating the limit, not merely choosing the most fluent interpretation of a number.

How do you test tool-result contracts?

Start with a successful empty collection as a positive case. Then interrupt a later page, repeat a page, change the advertised size, and remove fields required to interpret a record. Assert both the result state and the absence of authoritative totals when coverage fails. A test that checks only an error message can miss an accidental zero elsewhere.

Add a negative control that discards the error, and verify that the contract tests notice. Also retain the mixed-state example: passing all pagination checks must not be presented as proof of snapshot isolation. Our agent harness discussion provides the broader execution context, but each evidence boundary needs its own assertion.

This replay does not establish end-to-end model behavior, live database consistency or deployment status. It establishes how particular application functions interpret controlled inputs. The next check is whether the transport, consumer and final answer preserve those distinctions. “Nothing found” becomes useful only when the system can explain what it successfully looked at.