Free the models: harness design at the frontier

Model routers are everywhere right now, but they have a fundamental limitation. No matter if based on advanced heuristics or a small model that reads each turn and picks which LLM to use, a router will always be less capable than the model it’s choosing for. Replit Agent lets the model decide instead.

The main agent, or core loop, chooses its subagents’ tier and effort, and adjusts its own as the task unfolds. Given that freedom, GPT-6 Astra hands routine implementation to less costly subagents and decides for itself where its tokens are worth spending. On both DeepSWE and Terminal-Bench, Replit Agent is Pareto-efficient against Astra on its own: no published Astra baseline costs less and scores higher. It also beats a sidekick architecture, the same setup with one long-lived worker, by 11 and 16 points.

For the full post with animations and footnotes, check https://replit.com/blog/free-the-models (opens in new tab)

Why we scaffold less

Every model release invalidates assumptions baked into the harness.

As models become stronger at long-horizon tasks, they don’t need as much scaffolding at the harness layer. In practice, we’ve observed them lean more towards delegation on their own: using subagents for context management and parallelism. Recent breakthroughs, Navier–Stokes among them, came in part from coordinating swarms of agents powered by frontier models [1] (opens in new tab).

But the frontier is jagged. The strongest coding model is not necessarily the strongest at designing UIs or making slides, nor the best at writing emails. So we design our harness to let each model work its own way, with the guardrails it still needs and quality at minimum cost as the goal.

Each new model sends us back to re-test what we held firmly, and to experiment fast with techniques that build on emergent behaviors. Freeing the model, then, means letting it decide how hard to think, when to hand work off, and who to hand it to.

Composable primitives for delegation

When we started experimenting with GPT-6 Astra [2] (opens in new tab), we found that the model delegates well. The GPT-6 family is also the first from OpenAI to support effort changes mid-turn without breaking the cache.

To use these capabilities, we refined four harness primitives. They give the core loop a small set of choices at each step: what kind of subagent to dispatch, at what size and effort, whether to return to one it has already briefed, and how hard to think:

• Domain-aware subagents. Alongside a general worker, the harness offers specialists: read-only explorers, browser testers, reviewers, and a design subagent for slides and UI, each with its own model and tooling. For now, the harness still decides which specialists exist; the core loop decides when and how to use them.

• Subagent tiers and effort. Small, standard, and large, each a step up in cost and capability, and an effort level within the tier. Both apply to every subagent, and the core loop picks them at each dispatch. For example, a mechanical rename goes to small at low effort, while generating hypotheses for a stubborn bug goes to large at high effort.

• Reusable subagents. The core loop can return to a subagent it has already briefed instead of starting over. There is no single sidekick kept alive for the session: any number of subagents stay warm across kinds and tiers, and it picks which to wake. A longer cache lifetime on OpenAI’s newer models keeps the cost of doing so down.

• Dynamic effort tuning. Now that changing effort mid-turn preserves the cache on some models, we trained an escalation system that checks the trajectory at each step and matches effort to task difficulty. Unlike a router, it acts mid-turn on the work in progress, not once on the request.

The code quality of Astra and Fable 5.1 [3] (opens in new tab) also let us use our code-review subagent less, with no drop in our eval scores. We’ve not seen this level of engineering quality from any model before.

Newer models delegate on their own

Frontier models like Astra and Fable cost more per token, which makes them look uneconomical next to smaller ones. We’ve observed them naturally delegate to less costly subagents, keeping their own tokens for the decisions that need them.

Replit Agent never forces the core loop to spawn subagents. Table 1 shows how three models handle that decision in production:

All three models delegate, but each in its own way. At medium effort, Fable models rarely hand work to a general worker: they send out read-only explorers and reviewers and keep the implementation for themselves. Astra is the first model we’ve seen routinely delegate to general workers without being told to, and once it has briefed one it tends to go back to it rather than start over. This return rate has risen with every model generation.

Results

We evaluated Replit Agent in Max mode, our highest-quality setting with Astra as the core loop, on two software engineering benchmarks: DeepSWE and Terminal-Bench. We compare against two baselines: Astra on its own in mini-swe-agent, as published on each leaderboard, and a sidekick architecture, the same configuration with one change: its subagent primitives replaced by a single long-lived worker. Each chart plots score against cost per task, so the most efficient configurations sit toward the top left.

On DeepSWE v1.1 [4] (opens in new tab), which tests long-horizon changes to active open-source repositories, Replit Agent scores 72% at $2.11 per task. Astra in mini-swe-agent at low effort scores 67% at $1.60, and at xhigh effort 74% at $4.43; the sidekick architecture scores 61% at $1.34. Terminal-Bench 4.0 [5] (opens in new tab) tests multi-step work done entirely from a shell. Replit Agent reaches 49% at $2.53 per task, against 42% at $2.25 for Astra at low effort and 60% at $5.86 at xhigh. The sidekick architecture manages 33% at $1.84.

Replit Agent beats the sidekick architecture on both benchmarks, by 11 and 16 points. The sidekick costs less, and gives up a sixth to a third of the score for it. Astra on its own scores higher only by spending more: its best settings sit 2 and 11 points above Replit Agent at more than twice the cost. Neither baseline wins on both cost and score. We ran Replit Agent exactly as it ships to users, with no changes to the prompting or harness.

The bitter lesson of harness design

We read these results as an instance of Sutton’s bitter lesson [6] (opens in new tab). Baking human knowledge into an agent helps in the short term, plateaus in the long run, and is eventually overtaken by general methods that scale with computation. A rigid harness forces the model into one way of working; a composable one lets it choose. The smarter models get, the less the harness should decide for them.

Compared with a more prescribed architecture, this approach buys us three things:

• It bets on model scaling laws. Delegation that relies on the taste of the model improves with every release. Early previews of next-generation models continue the trend.

• It fits the task. The model spawns nothing for a small task, one explorer for a search, and a team when a build breaks into independent pieces.

• It reuses without persisting. A subagent keeps its context in case the model wants it back, and nothing persists unless it does.

In Sutton’s terms, the harness should let the model discover how to execute the work, not prescribe how we would have done it. Free the models.

Acknowledgements

Written by Daniel Furman, Jacky Zhao, Vaibhav Kumar, Ed Sioufi, and Michele Catasta. Thanks to James Austin, Toby Ho, Preeya Kirani, Zhen Li, Robin Newhouse, Devanshu Sen Pandey, Ibrahim Sheikh, Samuel Spitz, Peter Zhong, and the rest of the AI team at Replit for their contributions to this work. If you want to work on AI at Replit, my team is hiring; reach out to pirroh@repl.it.

References

1. On the Navier–Stokes Millennium Prize Problem (opens in new tab)

2. Introducing GPT-6 Astra (opens in new tab)

3. Introducing Claude Fable 5.1 and Claude Mythos 5.1 (opens in new tab)

4. DeepSWE v1.1 (opens in new tab)

5. Terminal-Bench 4.0 (opens in new tab)

6. The Bitter Lesson (opens in new tab)

7. Idiosyncrasies in Large Language Models (opens in new tab)

8. Design Arena (opens in new tab)

9. Terminal-Bench 4.0 results, Artificial Analysis (opens in new tab)