We keep giving agents tools, memory, permissions, multi-step goals and access to actual systems.
Then we test them with a spreadsheet of prompts.
At some point this starts to feel like testing a self-driving car with a multiple-choice exam.
For a simple LLM feature, the traditional eval makes perfect sense:
Input → model → output → score
Did it answer correctly? Was the tone right? Did it hallucinate? Great.
But an agent is not just producing an answer anymore.
It might check an account, call three APIs, update a CRM, ask the user for clarification, change its plan based on a tool response and then decide whether it has enough information to issue a refund.
The interesting part is no longer just what the model said.
It is what happened.
Imagine testing a customer support agent.
You give it this scenario:
A customer wants a refund for an order delivered 34 days ago.
You can evaluate whether the final response sounds reasonable.
But that tells you surprisingly little.
What if the refund API returns an error? What if this customer has different terms? What if another tool says the order was never delivered? What if the customer changes their story halfway through?
Or my personal favorite: what if the agent correctly explains that it cannot issue a refund, and then issues one anyway?
Once agents take actions, reliability starts depending on the world around them almost as much as the instructions inside them.
Anthropic makes a useful distinction in its 2026 guide to agent evals between the transcript, everything the agent did along the way, and the outcome, the actual state of the environment when it finished [1].
Their example is simple and important: an agent saying that a flight was booked is not proof that the task succeeded. The reservation actually existing is.
This sounds obvious.
Most testing infrastructures have not entirely caught up with the obvious part.
I think this is the bigger shift happening in agent evaluation.
Instead of defining only:
prompt + expected behavior
we increasingly need something closer to:
scenario + environment + actors + events + expected outcome
Now the test can include a sandboxed database. Tools can behave like the real tools without touching production. A simulated customer can react differently depending on what the agent does. An API can time out. Permissions can change halfway through. Another agent can make a conflicting update.
And, importantly, the exact weird situation that broke your agent yesterday can be replayed against the version you want to ship tomorrow.
At that point, we are no longer just asking agents questions.
We are putting them in situations.
This direction is already visible in research.
The original τ-bench benchmark puts conversational agents into simulated real-world domains where they interact with a simulated user, API tools and domain policies. Success is evaluated against the resulting database state, not just whether the conversation sounds good [2].
That is a meaningful change in what an eval actually is.
Its successor, τ²-bench, pushes the idea further. The researchers argue that many simulations still make the user unrealistically passive. In τ²-bench, both the agent and the user can act on a shared, dynamic environment [3].
In other words, even the simulated world needed to become more realistic because agents were getting too complicated for the old one.
BrowserGym follows a similar philosophy for web agents. Rather than reducing web-agent evaluation to a fixed question set, it provides standardized interactive environments with observation and action spaces [4].
Different domains, same trend.
The benchmark is slowly turning into a world.
I used to hear “digital twin” and immediately think of factories, jet engines and enterprise diagrams containing an irresponsible number of arrows.
But the concept fits agent systems surprisingly well.
A digital twin does not need to recreate every atom of the real system. It needs to reproduce enough of the important state and behavior that you can safely ask:
What happens if I do this?
For an agent, that could mean recreating the parts of reality that actually influence its decisions:
Not a copy of production. A controllable version of reality.
There is already research connecting these two areas. A 2026 paper on agentic systems and digital twins describes the digital twin as a shared situational model that maintains state, simulation capabilities and policy constraints while agents reason and act around it [5].
The paper is about cyber-physical systems, not customer-support agents.
But the architecture feels strangely familiar.
The more autonomy we give software, the more valuable it becomes to have somewhere realistic for that software to make mistakes without ruining anyone's Tuesday.
There is another consequence of this shift that I think is even more interesting.
Historically, testing and production were fairly separate worlds.
You wrote your tests. Then you deployed. Then production found all the tests you forgot to write.
Agents give us the opportunity to close that loop much more aggressively.
A weird tool sequence happens in production? Capture it.
An agent gets completely confused by an unusual customer? Scenario.
A particular permission setup sends it down the wrong path? Replay it.
Two agents somehow manage to get into an argument with each other and never finish the task? Congratulations. You have a regression test now.
Over time, your simulation environment can become a compressed collection of all the strange things reality has already thrown at your agents.
There is a nice example of this idea taken much further in FutureSim, a 2026 benchmark that evaluates agents by chronologically replaying real-world events. Agents receive information as it would have become available at the time and have to adapt as the simulated world progresses [6].
It is a very different use case, but I like the underlying principle:
Reality already generated some excellent test data for us. We should probably use it.
The mature version of this loop looks something like:
production → interesting behavior → recreate the state → replay → test changes → add to regression coverage → deploy
Then repeat forever, because apparently we chose software as a profession.
Quite the opposite.
We still need to decide what “good” means.
We need graders for policy compliance, task completion, safety, quality and all the beautifully subjective things humans eventually turn into a score between 0 and 1.
But I think the eval increasingly becomes one component inside something bigger.
The eval is the assertion. The simulation is the test.
You need a harness that can create the situation, control what the agent sees, give it realistic things to interact with, observe the entire trajectory and inspect what actually changed at the end.
Anthropic now explicitly recommends teams “build a robust eval harness with a stable environment” when evaluating agents [1].
I suspect that sentence is going to age well.
Because the first generation of AI testing was mostly about evaluating answers.
Agents are forcing us to evaluate behavior.
And behavior needs a world.
[1] Anthropic, “Demystifying evals for AI agents,” January 2026.
A practical guide to agent evaluation covering eval harnesses, environments, transcripts, outcomes, graders and the relationship between pre-production evals and production monitoring.
Anthropic: Demystifying evals for AI agents
[2] Shunyu Yao, Noah Shinn, Pedram Razavi, Karthik Narasimhan, “τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains,” 2024.
arXiv:2406.12045. Introduces simulated user-agent interaction with domain-specific tools and policies, with evaluation based on the final database state. (arXiv)
[3] Victor Barres, Honghua Dong, Soham Ray, Xujie Si, Karthik Narasimhan, “τ²-bench: Evaluating Conversational Agents in a Dual-Control Environment,” 2025.
arXiv:2506.07982. Extends agent simulation into environments where both the user and agent can take actions that modify shared state. (arXiv)
[4] Thibault Le Sellier De Chezelles et al., “The BrowserGym Ecosystem for Web Agent Research,” 2024.
arXiv:2412.05467. Introduces a standardized interactive environment with defined observation and action spaces for evaluating web agents. (arXiv)
[5] Agus Hasan, Dong Trong Nguyen, “Integrating agentic AI and digital twins for intelligent decision-making systems,” Array, Volume 29, 2026.
DOI: 10.1016/j.array.2026.100721. Explores an architecture where a digital twin provides shared state, simulation capabilities and policy constraints for autonomous agentic systems. (ScienceDirect)
Agentic AI and Digital Twins paper
[6] Shashwat Goel et al., “FutureSim: Replaying World Events to Evaluate Adaptive Agents,” 2026.
arXiv:2605.15188. Evaluates agents in grounded simulations created by chronologically replaying real-world events. (arXiv)
.png){kind=avatar}
.jpg)
.jpg)
