ZeroNoise Logo zeronoise
Post
Astra’s 60% Spend Lesson: Route Models by Task Complexity
5 min read
179 docs
Databricks’ Astra rollout turns model choice into a budgeted routing problem: its edge appears on hard system-design work, not routine coding. The rest of the signal is about making that distinction real with review gates, middleware, approvals, and transparent evaluation.

🔥 TOP SIGNAL

Astra’s advantage is concentrated, not universal. Databricks rolled Astra to ~3,500 engineers after a ~200-user pilot; it says Astra clearly outperformed Opus 5 and Sol 5.6 on highly complex work—especially high-level system design and long-range tasks—but did not clearly improve medium/low-complexity coding. Engineers using it increased coding spend by ~60%. The note also says Astra-vs-Fable comparisons are not robust because Fable was not widely rolled out under data-retention policies.

The practical pattern is the control plane: give the expensive model a separate sub-budget for hard tasks, default routine work to cheaper models, let engineers mix tools and models inside an overall budget, and revisit those limits.

⚡ TRY THIS

  • Copy the difficulty-based routing budget. Start with a cohort pilot, measure quality and spend, reserve the frontier model for system design and long-horizon work, and keep everyday coding on cheaper models. Do not make “best model” the default policy; make it an explicitly funded exception.

  • Choose bash or typed tools based on the boundary you can enforce. A Microsoft-paper summary in Latent Space reports that bash alone beat typed tool catalogs by 21.8–24.5 points on TheAgentCompany and 4.8–7.4 points on APEX-Agents while using fewer tokens. Use that as a design hypothesis: give agents bash inside a strong sandbox; use fixed programmatic tools when compliance requires a constrained inventory.

  • Split code review into passes, then add a UX gate. ThePrimeagen had Fable build a feature, Sol perform a thorough review, Grok remove unnecessary guardrails and superfluous code, and Sol check that simplicity did not break contracts. The result was still “one of the worst interfaces” he had seen. Keep the functional, simplicity, and contract passes—but require a screenshot, simulator, or user-flow check before calling the feature done.

  • Put privacy and tool auditing in middleware before adding autonomy. LangChain’s demo attaches prebuilt PII middleware to the agent’s middleware attribute, blocks email, redacts it on input before the LLM sees it, and verifies the lookup fails without exposing the value to the model or storing it in LangSmith. For an audit trail, create custom middleware with wrap_tool_call, attach it to the agent, and send the resulting tool logs to tracing or performance monitoring.

📡 WHAT SHIPPED

  • LangChain open-sourced open-paid-media-agent. The Slack Deep Agent runs every Monday across six ad platforms and a warehouse, explains what changed, proposes actions, and writes only after approval. The engineering pattern is unusually reusable: one graph with different Slack/cron capability profiles, task() delegating to one subagent per platform, Search → Read → Run tool discovery instead of loading a huge catalog, isolated report state, server-side user-ID permissions, approval cards, and post-write verification.

  • Claude collapsed Cowork and Chat into one Claude. Claude Design is now integrated so a conversation can produce a Slide, Design, or Doc; Claude decides whether to answer quickly or do deeper agentic work and selects the output format, while the user can stop or redirect it. The handoff model continues a report after the laptop is closed and asks for clarification when needed; rollout to Pro and Max is gradual. A firsthand user says the value is letting Claude decide how to get there while keeping attention on the work.

  • Kody v2026.09.16 adds password-manager-backed secrets behind a flag. Placeholders resolve at fetch time and remain invisible to the model; Kent C. Dodds specifically points to keeping secrets in 1Password or Bitwarden while allowing the agent to use them.

  • OpenWiki v0.5.2 adds a Kiro coding-agent integration. The setup is intentionally small: npm install -g openwiki@latest, then openwiki integrations install kiro.

  • Jev is an interesting control-plane model, not a GPT replacement. Latent Space’s roundup describes TypeSafe’s Jev/RLCD as optimized for decisions rather than text, with claimed 20–200× speed and 40–400× cost advantages; the important caveat is that it cannot produce free-form text and requires predefined output formats, making it a candidate classifier, judge, or router. Riley Brown reports roughly 1,000 email classifications in about 10 seconds, followed by informal tests of 500 classifications for 3.5 cents and 1,000 requests for 7 cents. Treat those as practitioner signals, not a coding benchmark.

  • Union Alpha is a model-transparency warning. OpenRouter advertises a free multimodal endpoint for coding and agentic workflows with 256K context and tool calling. Maria Ricks criticized providers for failing to disclose that it was a low-quality model router, and Theo agreed; until the serving behavior is documented, evaluate it as an opaque service rather than a clean model comparison.

  • Codex usage limits are not a dependable kill switch. NielsRogge says threads stop when the limit is hit; Theo says Codex still lets users continue in many cases. Put an external timeout or spend guard around unattended work and verify the process actually stopped.

🎬 GO DEEPER

  • Middleware for Managed Deep Agents — skip to the PII-redaction and tool-audit walkthrough. It is a compact implementation pattern for controlling what reaches the model and logging every tool call, rather than treating middleware as an abstract framework feature.
  • Underwriting Superintelligence, 00:23:46–00:25:03 and 01:20:05–01:22:20. Rune Kvist’s useful warning is that teams often have the right guardrail components but have not tested adversarial framings and corner cases. The later segment explains the harder coding-agent problem: one evolving red-team taxonomy that can cover Cursor, Harvey, and other long-horizon agents.

  • Study open-paid-media-agent. Focus on the source-of-truth rules, Search → Read → Run tool surface, per-subagent state isolation, and the approval/verification boundary—not the ad domain. Those are portable harness patterns for any agent that reads broadly but writes narrowly.

Editorial take: The practical edge is to route expensive intelligence to hard work, separate review dimensions, and put policy, approval, and adversarial tests around every action the model can take.

Astra’s 60% Spend Lesson: Route Models by Task Complexity
Summary
Coverage start
1 day ago
Coverage end
4 hours ago
Frequency
Daily
Published
3 hours ago
Reading time
5 min
Research time
3 hrs 45 min
Documents scanned
179
Documents used
21
Citations
32
Sources monitored
110 / 111
Insights
Skipped contexts
Source details
Source Docs Insights Status
Brent Traut 2 1
Lukas Möller 0 0
Jediah Katz 2 1
Aman Karmani 0 0
Jacob Jackson 0 0
Cursor Blog | RSS Feed 0 0
Nicholas Moy 0 0
Mike Krieger 3 1
Sualeh Asif 0 0
Michael Truell 0 0
Google Antigravity 0 0
Aman Sanger 0 0
cat 2 1
Mark Chen 0 0
Greg Brockman 6 2
Tongzhou Wang 0 0
fouad 0 0
Calvin French-Owen 0 0
Hanson Wang 0 0
Ed Bayes 0 0
Alexander Embiricos 0 0
Tibo 7 0
Romain Huet 3 1
DHH 42 3
Jane Street Blog 0 0
Miguel Grinberg's Blog: AI 0 0
xxchan's Blog 0 0
<antirez> 0 0
Brendan Long 0 0
The Pragmatic Engineer 0 0
David Heinemeier Hansson 0 0
Armin Ronacher ⇌ 9 1
Mitchell Hashimoto 0 0
Armin Ronacher's Thoughts and Writings 0 0
Peter Steinberger 0 0
Theo - t3.gg 30 5
Sourcegraph 0 0
Anthropic 0 0
Cursor 0 0
LangChain 1 1
Anthropic 0 0
LangChain Blog 0 0
LangChain 13 6
Cursor 0 0
Riley Brown 1 1
Riley Brown 11 4
Jason Zhou 6 2
Boris Cherny 4 1
Mckay Wrigley 0 0
geoff 3 1
Peter Steinberger 🦞 0 0
AI Jason 0 0
Alex Albert 2 1
Latent.Space 2 2
Logan Kilpatrick 0 0
Fireship 0 0
Fireship 0 0
Kent C. Dodds 🐨 15 3
Practical AI 0 0
Practical AI Clips 0 0
Stories by Steve Yegge on Medium 0 0
Kent C. Dodds Blog 0 0
ThePrimeTime 0 0
Theo - t3․gg 0 0
ThePrimeagen 2 1
Ben Tossell 4 2
swyx 0 0
AI For Developers 0 0
Geoffrey Huntley 0 0
Addy Osmani 0 0
Andrej Karpathy 0 0
Simon Willison 5 2
Matthew Berman 0 0
Changelog 0 0
Simon Willison’s Newsletter 0 0
Agentic Coding Newsletter 0 0
Latent Space 1 1
Simon Willison's Weblog 2 1
Elevate 0 0
Lukas Möller 0 0
Jediah Katz 0 0
Sualeh Asif 0 0
Mike Krieger 0 0
Michael Truell 0 0
Cat Wu 0 0
Kevin Hou 0 0
Aman Sanger 0 0
Nicholas Moy 0 0
Andrey Mishchenko 0 0
Jerry Tworek 0 0
Romain Huet 0 0
Thibault Sottiaux 0 0
Alexander Embiricos 0 0
xxchan 0 0
Salvatore Sanfilippo 0 0
Armin Ronacher 0 0
David Heinemeier Hansson (DHH) 0 0
Alex Albert 0 0
Logan Kilpatrick 0 0
Shawn "swyx" Wang 0 0
Jason Zhou 0 0
Riley Brown 1 1
McKay Wrigley 0 0
Boris Cherny 0 0
Ben Tossell 0 0
Geoffrey Huntley 0 0
Peter Steinberger 0 0
Addy Osmani 0 0
Simon Willison 0 0
Andrej Karpathy 0 0
Harrison Chase 0 0