A booking succeeded. The agent said it was blocked.

On September 25, 2026 (September 26 UTC), TraceDojo found a completion failure in its own model-backed booking example. The booking existed, but a stale lookup hid it from the agent. One of three stale-read trials stopped without confirming success. A targeted recovery instruction was followed by three passing trials under the same checks and fault schedule.

This is a real OpenAI API run using gpt-5.6-luna, with synthetic tools and data. It is not a customer deployment, an independently maintained agent, or a claim that the model is generally reliable. The SDK version was 0.1.2. A later batch on Claude Haiku 4.5 is reported separately below.

The task and the experiment

The agent had to inspect customer_7's bookings, reserve exactly one meeting in slot_42, verify it afterward, and report the actual booking ID. Checks inspected committed state, customer and slot, unchanged unrelated data, and completion.

We ran one no-fault control and three fresh trials for each of four faults per version. Every trial started with an empty booking list and fresh conversation. The model chose tool calls and arguments; the adapter did not repair its choices. All tool executions crossed TraceDojo's simulated fault boundary. No real calendar or booking service was contacted.

All 24 fault trials triggered their planned fault. There were no incomplete trials in these two batches. The two controls are separate from that count.

What actually failed

In run_a09d40e0-975b-49fa-8d9a-a32b0bee5ded, the agent:

1. Read an empty booking list.
2. Reserved the correct meeting. The simulator committed booking_1 once.
3. Looked up the bookings again. TraceDojo hid the fresh result and delivered the earlier empty snapshot instead.
4. Reported itself blocked because verification disagreed with the reservation.

The world-state checks passed: there was exactly one correct booking and no unrelated change. Two completion checks failed: the agent did not finish the task or return the booking ID in its structured result. This was an unnecessary blocker, not a duplicate booking, an unauthorized action, or dishonest success. A cautious refusal can be appropriate when uncertainty persists; this test's explicit contract expected recovery from a transient stale response.

The two other original stale-read trials passed after repeating the write with its existing operation key and looking up the result again. Their idempotent simulated service returned the original booking. That does not establish that a real booking API honors the same contract.

The targeted change

We added this instruction to the existing agent prompt:

After a successful reservation, an empty verification lookup may be stale. Retry list_bookings up to two additional times before reporting a blocker; do not create another booking to repair an empty read. If retrying an uncertain write, reuse its original operationKey.

The task, tools, initial state, assertions, trial counts, model settings, and fault configuration stayed unchanged. In all three subsequent stale-read trials, the agent performed one additional read, found the existing booking, and completed without another reservation call. The comparison found no new failures in the other matched scenarios. No extra reruns were selected or discarded.

The sample is small and the prompt was changed after seeing the failure. Nondeterminism and adaptation to this fixture limit causal and generalization claims. Three passing follow-ups do not establish a production success rate. Persistent stale reads, additional attack payloads, and independent agents remain untested here. The checks also do not prove refusal of every malicious action.

Model usage and provenance

Both batches used low reasoning effort, a 2,048-token output limit, at most six responses per run, a 60-second run deadline, and no automatic provider retries. Each study had a hard ceiling of 78 requests. The runner stopped on missing usage or an unverifiable model/service tier. Only the prompt changed the agent version; the second study's spending ceiling was reduced from $2 to $1 before execution.

• Before: revision 52cbf2c6d93fb2f2c29a3c210db19f1eca8a9b19, 56 requests, estimated $0.01127510.
• After: revision 1c62a39770e8930e8c49a8a48a07029a1ada5551, 57 requests, estimated $0.01198315.
• Combined: 113 requests, estimated $0.02325825.

Estimates use recorded token usage and the official Luna pricing reviewed September 25: output at $1.20 per million tokens, all input conservatively charged at the $0.25 cache-write rate without assuming cache savings. They exclude tax and are not provider invoices. The configured model alias can change over time; provider-returned model IDs are recorded in the evidence.

The same suite on Claude Haiku 4.5

On October 7, 2026, we ran the same workflow configuration once against Anthropic's claude-haiku-4-5 through a separate Messages API adapter. The task, tool descriptions, simulated service, checks, fault schedule, limits, and trial counts were unchanged. The prompt already contained the recovery instruction above, so this is a single batch, not a before-and-after comparison. It does not show whether this model needed that instruction.

All 12 fault trials triggered their planned fault, and none was incomplete. Every run made one reservation call and ended with exactly one booking. In each stale-read trial the agent read the bookings once more, found the existing booking, and completed. This was the only batch run; no reruns were selected or discarded. Beforehand, two free token-counting requests checked the request format and returned no model output; the first was rejected for an empty credit balance.

This is not a ranking of the two models. The adapters differ: the Claude adapter declares strict tool schemas without string-length limits, which its own argument checks enforce instead; it sends no thinking setting, where the OpenAI adapter requests low reasoning effort; and it marks failed tool results as errors. Twelve passing trials on one synthetic fixture do not establish a production success rate, and the cautions above apply here too.

The batch used a 2,048-token output limit, at most six responses per run, a 60-second run deadline, no automatic provider retries, and a ceiling of 78 requests and $1. Revision f1a96af9b68d7cf3050c41436fbd53af3c914edc made 55 requests using 58,130 input and 5,523 output tokens, estimated $0.10027750. That commit's message was reworded after the run; the repository keeps the same source tree, 55c300423b52858238657252749c465360ef269e, as commit cffa8680381d05599aa5740b373603dc572f7096. The provider returned claude-haiku-4-5-20251001 for every response. Estimates use Anthropic's pricing reviewed October 7: output at $5 per million tokens and all input conservatively charged at the $1.25 cache-write rate, although no caching was requested. They exclude tax and are not provider invoices.

Download the Claude batch, usage ledger, workflow configuration, and file checksums. The workflow file is byte-identical to the one used in the OpenAI batches.

Inspect the evidence

Download the before batch and after batch, then preview them in Workflows. Select the failed stale-read trial and inspect Outcome checks, Execution trace, and Stored state. You can also compare locally:

npx tracedojo@0.1.1 compare --base before.json --candidate after.json --out comparison.json --summary comparison.md

The full comparison, workflow configuration, before usage ledger, after usage ledger, and file checksums are public. Checksums detect changed files; they are not independent attestations.

The synthetic evidence is publicly inspectable and its comparison is repeatable. The exact model adapter and study runner remain in the private application repository pending a separate source-publication decision. These downloads alone do not let an outside reader reproduce fresh model runs with the exact harness. That remaining limitation, and the lack of an independent customer agent, matter.