Researchers demonstrate a class of black-box attacks called Repeat-After-Me (RAM) that embed adversarial perturbations inside images to hijack commercial vision-language models and the deployed agent OpenClaw. The attacks achieve measurable success rates against real systems without access to model weights, making them practical threats rather than theoretical concerns. The paper quantifies how an attacker can steer agent behavior, extract information, and cause the agent to take unintended actions, all by controlling a single image the agent sees.
The design-threat angle that matters most for infrastructure work is TOOLS.md overwrite: an injected image can cause an agent to rewrite the file that defines its available tools and permissions. This is a context-integrity attack on the agentic layer itself, not the application layer. An agent that trusts its tool definitions from a file it can edit is vulnerable to having its entire permission scope silently replaced by a malicious image in its visual context.
For designers and engineers building agentic systems, this paper makes the case that approval gates, permission scoping, and observability are not optional polish. An agent that can act on injected instructions without a human seeing the intermediate steps has no recoverable trust boundary. The combination of background execution and vision input creates an attack surface that current UX patterns address poorly.