← Field Notes
OCT 5 · Release · via GitHub Changelog ApprovalReach PreviewRecovery

GitHub Copilot learns to click and type in desktop apps

Copilot now acts in software the developer did not write. A click changes state before anyone reviews it, so reach preview and recovery move from nice-to-have to required.

Machine summary of the source

GitHub has added computer use to Copilot, its AI coding assistant. The agent can now move a mouse, click buttons, and type in desktop applications on the developer's machine. This moves Copilot from a tool that suggests code inside an editor to one that can operate software on your behalf. That shift changes the stakes. Asking Copilot to write a function carries low risk: you read the code before it runs. Asking Copilot to click through a desktop application means the agent may change state in software before you see what it did. The gap between what you asked and what happened becomes harder to close. Reversing a mistake may require more than deleting a line. For teams designing agent products, this is a clear signal that approval, recovery, and reach preview patterns are no longer edge cases. A developer handing off a desktop task needs to know what the agent will touch before it starts. They need a chance to stop it mid-run, and a clear account of what it did after. Without those surfaces, the interaction asks for trust the product has not yet earned.

The summary above is generated; the note at the top is the editorial judgment. Primary source ↗