Kolab measured 115 hours of real work and found agents need humans

Kolab is a new test suite built from over 115 hours of professional screen recordings. Researchers captured real knowledge workers doing real tasks, then built the tests around what those workers actually did, including when they stopped, redirected, or corrected the work in progress. Most existing test suites assume the agent runs to completion on its own. Kolab shows that real work does not look like that. The recordings reveal a pattern: humans step in, check progress, and change direction at key moments. The test suite treats those moments as first-class events, not interruptions to be designed out. A product team that trains or tests an agent only on solo runs will miss the behaviors that matter most in a professional setting. For designers, the core finding is structural. The right question is not whether an agent can finish a task alone, but whether a human and an agent can finish it together. Kolab gives teams a way to measure that. It names the check-ins, the redirections, and the correction steps as measurable parts of quality, not noise.