← Field Notes
NOV 4 · Paper · via arXiv Agent IdentityApprovalGenerative UI

Interface Quilt adds buttons to live apps without touching their code

The agent writes controls into the interface and the human sees the result. No preview, no marking, no visible seam. The trust problem in generated interfaces is the silence around who built what.

Machine summary of the source

A research paper introduces Interface Quilt, a system that pairs an AI with generated user interfaces to modify running applications without touching their source code. The agent observes the screen, reasons about the task, and stitches new controls into the interface. The result is a live, hybrid interface assembled on the fly rather than shipped by a developer. For a UX leadership audience, the central tension is trust. The human sees controls the agent built, but those controls carry no prior design review, no brand vetting, and no accessibility audit. The agent-built layer is indistinguishable from the product layer unless the team marks it. That unmarked seam is a design problem before it is a technical one. The paper also surfaces a sequencing question: does the agent show the human what it plans to add before adding it, or does it act and present the result? Interface Quilt favors action first. Teams building similar systems will need to decide whether the human approves the proposed control, reviews it after insertion, or trusts the agent to judge the fit. Each choice produces a different relationship between the human and the modified interface.

The summary above is generated; the note at the top is the editorial judgment. Primary source ↗