ZeroNoise Logo zeronoise
Post
Verification is the bottleneck: Claude Code's 90% self-authorship, Huntley's Antithesis loops, and LangChain's agent cost controls
•
6 min read
• 122 docs
Anthropic credits verification for letting Claude write about 90% of Claude Code's code, and Huntley is putting Overalls through two weeks of autonomous verification. Also covered: LangChain's method for controlling coding-agent spend, Huntley's composable inference proxies, credential hygiene, and smaller tool updates.

Verification, not review, is what lets agents write most of the code

Anthropic says Claude writes about 90% of Claude Code's code. From June to September, merged PRs more than doubled while the share of PRs reverted fell by more than 40% . Addy Osmani puts this down to one practice: give the agent a way to verify its work. When code is produced faster than people can review it, the bottleneck becomes proving that it works .

His recipe:

  • Replace the single reviewer with layers. One person reading line by line can no longer be the gate. Use a multi-layered system with fast, deterministic feedback loops for agents: end-to-end tests, feature maps (which poteto advocates), property tests and multi-agent adversarial reviews .
  • Move humans to judgement. People should focus on product intent, assessing risk and weighing evidence. Leave code checking to automated tests, scale approvals to risk, and turn test failures into living evaluations .

Anthropic's write-up is here . One caution from the field: in a Reddit thread where a developer said Opus 5.5 cut their tickets to 1–2 hours a day, a commenter said manual review on their team still catches "significant issues, including data loss and security issues" .

Geoffrey Huntley is testing the same idea under stress. His starting point: just because an application compiles doesn't mean it works. Overalls has gone into Antithesis's "torture chamber", where he will run autonomous discovery-and-rectification loops for two weeks until his New York talk. He wants to see what classes of problems turn up and whether agents with "the right level of back pressure" can fix failing properties on their own . He also flagged a slide deck on formal verification of LLM-generated code .

He also proposed using thinking traces to check whether a skill or AGENTS.md is working: whether the agent considers the skill, and when it decides not to use it. He notes this isn't possible with Claude .

Controlling coding-agent spend: LangChain's three steps

Harrison Chase guesses LangChain spends roughly 30% of its engineering salaries on coding tools. That share has fallen over the last two months even though token use went up . His steps:

  1. Visibility. Log all coding-agent usage to LangSmith so you can see who spends what, and on what .
  2. Basic controls. Set budgets with a way to escalate: the VP of engineering almost always raises the limit, but the budget guards against agents stuck in bad loops .
  3. Own the harness. Their in-house cloud agent, which Chase starts from Slack, has prompt caching and model routing built in . Providers now let you add tools mid-run, which used to break the cache .

The example that justifies tracking: a plugin sent a DOM snapshot back on every agent turn during frontend work, and that one loop cost about $40k in a week .

Prompt caching is also behind Theo's claim that Gemini is about 50% more expensive than it should be. His test showed a 27% cache-miss rate , using the same basic agentic-work test on all three APIs . A respondent said their own miss rates are much lower . Theo says the test is about 10 lines of code and published his methodology , so you can check your own workload.

Huntley's composable inference proxies

Huntley has lost time to "model is at capacity" errors that start when Europe wakes up, so he released three pieces of software that can be used separately or together :

  • Underclass pools ChatGPT subscriptions behind a proxy, claims usage resets automatically and handles the prompt cache key so usage isn't wasted. He calls it stable, runs it with six ChatGPT 500 accounts, and it's on GitHub and available as a Nix Flake .
  • Preflight is a local proxy that scans requests and attachments and redacts secrets before they reach the provider. Point Underclass at Preflight to use both .
  • geoclass is an experiment. Tag VMs around the world on a Tailscale network, point Underclass at geoclass, and requests get routed across regions. "In theory" the rest just works .

Early results are anecdotal. Claude found no header in the Codex source or OpenAI docs that pins requests to a region . After routing out of Germany via Australia, Huntley says OpenAI inference felt much better, but he hasn't measured it and it was a Saturday afternoon .

Keep secrets where agents can use them but not read them

A user blamed a $90,000 Google bill on a leaked key, and said Opus 5.5 found that Codex had published their .env to Cloudflare Pages. They say Codex itself flagged the chat as a cybersecurity risk . This is their account and hasn't been verified. Kent C. Dodds said he feels safer because he uses cloud agents and keeps secrets and OAuth tokens where agents can use them but not retrieve them . He also restricts tokens to specific actions, for example Gmail drafts only . Preflight, above, is another way to do this.

The opposite complaint also surfaced: auto mode in dots and Claude Projects refuses or misclassifies basic tasks, such as merging PRs into a repo set up for that purpose. This blocks work overnight or demands approvals you can't give remotely . Riley Brown agreed.

Smaller items

  • Claude Code latency: it now gzips the conversation it sends each turn, and cloud sessions start up as soon as you begin typing . Theo says responses now start almost immediately, and the difference is most noticeable when steering, because you can watch the model correct itself .
  • cc-stretch is an MIT-licensed Claude Code plugin. Replies start compressed and you expand sections as needed (it claims 83% fewer words read), and it tells Claude which sections you expanded. Install with /plugin install cc-stretch --marketplace pro-vi/cc-stretch and run /tui fullscreen .
  • Consistent harness runs: OpenHands looks stable mainly because it hardcodes temperature 0. Normalizing tool outputs fixes much of the variance in other harnesses .
  • Haiku 5.5 costs $0.10/$0.50 for prompts up to 100K tokens and is reported at 39.2% on Terminal-Bench 4.0 at max effort .
  • Linus Torvalds used an LLM to refactor hard-to-follow associative-array code into smaller functions. Userspace tests confirmed the reported bug and found three more, all dating to 2013, according to an Italian video recap . In the same video, Salvatore Sanfilippo says AI helped him write Redis patches that cut RAM use on large sorted sets by 50% .
  • Email as an agent channel: Ben Tossell's bot opens BB app threads for coding work, emails updates, and updates a shared task list and work log . He says email threads work well for agent chats .
  • Riley Brown edited a recorded video in one prompt with Opus 5.5, Blender and a skill he had worked on for a day. He rates it about 3% as good as it could be .
  • tsc-ts from Vercel Labs ports TypeScript back from Go to TypeScript, compiles it with scriptc, and claims 2–4x faster than tsgo . Theo says it beats his tsc-rs port in most things .
Verification is the bottleneck: Claude Code's 90% self-authorship, Huntley's Antithesis loops, and LangChain's agent cost controls