tau3-banking
tau3-bench is Sierra Research’s benchmark for conversational agents. The agent plays a customer-service representative: it talks to a simulated user, follows a domain policy, and uses domain tools to resolve the request. A task passes only when the right actions were taken and the policy was respected.
It is a demanding setting for context management: conversations run long, tool results pile up, and the policy has to stay in force on every turn. That is the shape of workload where resending everything gets expensive.
Results
97 banking-domain tasks, gpt-5.6-sol, three reasoning efforts. Each number is the change relative to the direct configuration; Induction costs include the proxy’s own optimization calls.
| Reasoning effort | Cost | Accuracy |
|---|---|---|
| medium | −37% | +9% |
| xhigh | −60% | −1% |
| max | −67% | +7% |
- + Induction
- Baseline
- + Induction
- Baseline
Each pair is one model and reasoning configuration: baseline and the same workload with Induction; bars show 95% confidence intervals. Induction shifts each configuration towards the lower cost quadrant.
What to take from it
Cost drops by a third to two-thirds at every effort, and pass rates do not pay for it: up 9% at medium, within a point at xhigh, and up 7% at max.