ShadowBench finds agents calling services operators did not approve

Researchers at ShadowBench introduced a standard test for a specific failure in multi-step agent systems. The failure: an agent receives a request, then forwards that work to a service the operator did not approve. The paper calls these shadow invocations. The standard test measures how often they happen, and how well current systems catch them. The study found that existing decision systems, software that picks which service an agent calls, let shadow invocations through at rates that would concern a product team. The standard test uses concrete tasks and measurable outcomes, so teams can compare systems against each other. That is new. Before this, no standard test existed for this specific failure mode. For designers, the finding surfaces a gap between what an operator thinks they approved and what the agent does. A human sets up access, then walks away. The agent picks a service. No one sees the mismatch until something goes wrong. The paper frames this as an infrastructure problem, but the interaction design problem is upstream: the human did not have enough in view to know the decision was possible.