ZeroNoise Logo zeronoise
Post
How the Claude Code team reviews agent-written code: more than twice the PRs merged, over 40% fewer reverts
•
5 min read
• 95 docs
Anthropic's Claude Code team describes its layered, mostly automated review pipeline and the limits it puts on review loops. Also: Theo uses Opus to fix T3 Code's UI, Gemini makes things up in agentic review, and LangChain shows how its engineering agent handles identity.

The Claude Code team's review pipeline

Addy Osmani shared a write-up of how the Claude Code team reviews code. Claude writes about 90% of Claude Code. From June to September, merged PRs more than doubled while the share of PRs that had to be reverted fell by more than 40% . The team still reviews code. Review now happens continuously while the code is written, so Claude has usually reviewed and corrected a change at least once before a human looks at it . Their goal is for most PRs to merge with at most one person in the loop, and they treat revert and incident rates as a guardrail .

Patterns you can copy:

  • Ask for proof before opening a PR. Have Claude run something, such as tests, the build, or a screenshot compared with the design: "Instead of saying the task is done, we ask it to show us the result." Then a subagent with fresh context reviews the diff. Many team members also run their own skills for bug hunting, shrinking the diff, end-to-end validation and checking UI against design guidelines . Boris Cherny estimates a verification process improves output quality by 2–3× .
  • Fix repeated findings at the source. If the reviewer keeps flagging the same issue, change the skill, add a lint rule, or update the docs. Keep CLAUDE.md short, since it is loaded into every review. In REVIEW.md, owners say what counts as failure, limit nitpicking, list paths to ignore, require file and line references, and ask for a top-level summary that includes a count and a clear "no issues" when there are none .
  • Filter findings with several agents. Agents hunt for bugs in parallel. A separate vote keeps only findings with enough support, and findings that are only about style or cleanup are dropped. The team A/B tested these filters on its own PRs. Comments are labeled Important, Nit or Pre-existing . These features ship as Code Review for Claude Code on Team and Enterprise plans, with a lighter local /code-review. Since the preview, the share of internal PRs getting substantive comments rose from 16% to 54% .
  • Cap review loops. One PR went back and forth between agent author and agent reviewer more than 50 times overnight. About 59% of what the reviewer kept flagging was code that had been there since the first pass . The fix: after three cycles, the reviewer asks the author to rethink the approach. Later rounds look only at the diff, name the class of bug, and don't repeat findings already reported .
  • Set human review by the change's risk, not the author. Classifiers decide the risk level. Sensitive paths and CI workflow files always need another approval. Agent approvals don't count as human approvals, and approval must be given again after untrusted commits .
  • Use past incidents as the eval set. The review system is tested daily against a collection of PRs that caused incidents. Noise is measured by how people respond to each comment .

Humans still decide whether to build something, audit dangerous code like billing and CI, review architecture, check the evidence, and answer for every change . On the same theme, Theo posted: "If you're still merging most PRs by hand, you're behind" . Armin Ronacher polled whether people still write more than 20% of their code by hand. He reads the results so far as showing the base is "divided" .

Theo: Opus fixes UI, and the hallucination record of another model

Robin Ebers criticized T3 Code's fonts, border radius, icons and font hierarchy, and pointed to Capy as inspiration . Theo asked whether he could turn the UI into "a hill climb for Opus" . He opened pingdotgg/t3code#24837 , later posted "Holy shit it worked" , and called Opus 5.5 "the greatest model of all time" . After a few hours on the build, he said he hadn't realized how much better a cleaned-up design would feel . If you have a vague design complaint, try turning it into a goal the model can iterate toward.

He also ran a thread on an Ubuntu machine that drove an iOS simulator on a Mac mini connected through T3 Code. He calls it "the best iOS dev experience I've ever had" .

Theo's council skill has models review each other's work. He now lists specific Gemini hallucinations from one run: a line number (line 173 cited in a 72-line file), IP addresses, command arguments, and function exports and returns . He calls it "a straight up liability in your stack" . These are his own observations, not a benchmark.

LangChain's OpenSWE: identity and sandbox design

LangChain's engineering agent, OpenSWE, drafts PRs and reviews every PR . When you message it, it acts as you and opens PRs under your GitHub handle. When it reviews in the background, it uses its own GitHub App identity . It picks a sandbox base image based on the channel or user. It runs outside the sandbox and executes code inside it, using a secret proxy to reach authenticated services . A new PR automatically triggers a review .

Shorter notes

  • Compaction (antirez). Salvatore Sanfilippo disagrees with commenters who say you should never reach context compaction. He says compaction comes up less often for him, and models are less disoriented afterward with Astra and especially Opus 5.5. The model now has an explicit tool to search the session's JSONL log; before, he had to tell it to look there .
  • Spending (antirez). He had switched his Anthropic plan from 20x to 5x by accident and, using Opus 5.5 heavily, never hit the weekly limit, so he kept it . He argues for flat subscriptions, for example a €200 reimbursement per employee, over paying API rates . He also argues that agents give the most leverage to developers with product vision who steer a project at a high level .
  • Ramp-up prompt (TheStandup). On a new codebase, ask the AI to find the relevant repos, explain why something behaves the way it does, and read the commits to trace the original decision. One host says this covers "at least a good 80%" of the questions that slow onboarding .
  • Keep your data portable. A user said Grok Bot deleted weeks to months of work for him and others . Kent C. Dodds responded by recommending Kody, which keeps your work outside any one walled garden so you can switch agents without a migration .
How the Claude Code team reviews agent-written code: more than twice the PRs merged, over 40% fewer reverts
Summary
Coverage start
2 days ago
Coverage end
1 day ago
Frequency
Daily
Published
1 day ago
Reading time
5 min
Research time
1 hr 35 min
Documents scanned
95
Documents used
19
Citations
36
Sources monitored
110 / 111
Insights
Skipped contexts
Source details
Source Docs Insights Status
LangChain Blog 1 1
Brent Traut 0 0
Lukas Möller 0 0
Jediah Katz 2 0
Aman Karmani 0 0
Jacob Jackson 0 0
Cursor Blog | RSS Feed 0 0
Nicholas Moy 0 0
Mike Krieger 0 0
Sualeh Asif 0 0
Michael Truell 0 0
Google Antigravity 0 0
Aman Sanger 0 0
cat 0 0
Mark Chen 0 0
Greg Brockman 2 0
Tongzhou Wang 0 0
fouad 1 0
Calvin French-Owen 0 0
Hanson Wang 0 0
Ed Bayes 0 0
Alexander Embiricos 0 0
Tibo 2 0
Romain Huet 0 0
DHH 8 1
Jane Street Blog 0 0
Miguel Grinberg's Blog: AI 0 0
xxchan's Blog 0 0
<antirez> 0 0
Brendan Long 0 0
The Pragmatic Engineer 0 0
David Heinemeier Hansson 0 0
Armin Ronacher ⇌ 4 1
Mitchell Hashimoto 0 0
Armin Ronacher's Thoughts and Writings 0 0
Peter Steinberger 0 0
Theo - t3.gg 46 8
Sourcegraph 0 0
Anthropic 0 0
Cursor 0 0
LangChain 0 0
Anthropic 0 0
LangChain 0 0
Cursor 0 0
Riley Brown 0 0
Riley Brown 4 1
Jason Zhou 2 1
Boris Cherny 0 0
Mckay Wrigley 0 0
geoff 3 0
Peter Steinberger 🦞 0 0
AI Jason 0 0
Alex Albert 0 0
Latent.Space 0 0
Logan Kilpatrick 0 0
Fireship 0 0
Fireship 0 0
Kent C. Dodds 🐨 8 2
Practical AI 0 0
Practical AI Clips 0 0
Stories by Steve Yegge on Medium 0 0
Kent C. Dodds Blog 0 0
ThePrimeTime 1 1
Theo - t3․gg 0 0
ThePrimeagen 4 0
Ben Tossell 0 0
swyx 3 0
AI For Developers 0 0
Geoffrey Huntley 0 0
Addy Osmani 2 1
Andrej Karpathy 0 0
Simon Willison 0 0
Matthew Berman 0 0
Changelog 0 0
Simon Willison’s Newsletter 0 0
Agentic Coding Newsletter 0 0
Latent Space 0 0
Simon Willison's Weblog 0 0
Elevate 0 0
Lukas Möller 0 0
Jediah Katz 0 0
Sualeh Asif 0 0
Mike Krieger 0 0
Michael Truell 0 0
Cat Wu 0 0
Kevin Hou 0 0
Aman Sanger 0 0
Nicholas Moy 0 0
Andrey Mishchenko 0 0
Jerry Tworek 0 0
Romain Huet 0 0
Thibault Sottiaux 0 0
Alexander Embiricos 0 0
xxchan 0 0
Salvatore Sanfilippo 2 2
Armin Ronacher 0 0
David Heinemeier Hansson (DHH) 0 0
Alex Albert 0 0
Logan Kilpatrick 0 0
Shawn "swyx" Wang 0 0
Jason Zhou 0 0
Riley Brown 0 0
McKay Wrigley 0 0
Boris Cherny 0 0
Ben Tossell 0 0
Geoffrey Huntley 0 0
Peter Steinberger 0 0
Addy Osmani 0 0
Simon Willison 0 0
Andrej Karpathy 0 0
Harrison Chase 0 0