← Field Notes
NOV 4 · Paper · via arXiv Budget ScopingContextPermissions

CLEO cuts the model size 100x and still passes 83.8% of tests

CLEO shifts the work from picking a bigger model to making deliberate scoping decisions before the agent starts. For a product team, that means permissions and budget limits are now primary design work, not configuration.

Machine summary of the source

A research team built CLEO, a minimal investigative analysis agent, to test a counterintuitive idea: that tuning the scaffolding around a small, self-hostable language model produces better results than scaling the model itself. The agent ran on a model roughly 100 times smaller than the frontier alternatives and achieved an 83.8% pass rate on its benchmark suite. The finding shifts where a product team should invest effort — from acquiring a larger model to designing tighter, more deliberate scaffolding.

The core design move is treating the scaffolding, the set of instructions, tools, and boundaries given to the agent before it starts, as the primary site of improvement. CLEO demonstrates that a well-fitted scaffolding can compensate for a model's raw capability limits. This has direct consequences for teams working under data-residency constraints or cost ceilings, where a self-hosted, smaller model is a requirement and not a choice.

For designers, the practical tension is in scoping and trust. A tighter scaffolding means the team must decide in advance what the agent can reach, what it can do, and how far it can go. Those decisions are design decisions, made before the agent runs. CLEO's results suggest that getting those decisions right matters more than the model behind them.

The summary above is generated; the note at the top is the editorial judgment. Primary source ↗