Blind participants exposed what computer-use agents still cannot do

A study pairing real blind users with a novel prototype reveals a largely ignored failure mode in computer-use agents: the assumption that a sighted, visually-scanning interaction model is the default. Agents that navigate desktop UIs by screenshot and pixel-level action produce outputs and feedback loops that are opaque or entirely inaccessible to screen-reader users — not just inconvenient, but structurally broken.

The research surfaces specific breakdowns around progress feedback, action confirmation, and error recovery — moments where a sighted user can glance and self-correct, but a blind user has no equivalent signal. The prototype tested here attempts to thread accessible feedback into the agent's action loop, revealing both what is possible and how much the default architecture works against it.

For UX designers, the implication is blunt: if your agent's legibility model is 'the user can see what it's doing,' you have an accessibility debt baked into the interaction pattern itself, not just the surface. Progress streaming, confidence signals, and interruption hooks are not enhancements — they are load-bearing for any user who cannot visually monitor agent behavior in real time.