We can't find the internet
Attempting to reconnect
Something went wrong!
Hang in there while we get back on track
🔥 TOP SIGNAL
Claude Code Opus 5 Auto Mode should be treated as approval UX, not containment. Johann Rehberger’s targeted test, summarized by Simon Willison, used a ZIP archive’s local struct.py to shadow Python’s standard library; variants succeeded in 3/5 or 4/5 runs, but the author explicitly calls those small samples rather than a universal attack-success rate. In some runs Claude recognized the compromise but Auto Mode denied cleanup; the report records Anthropic’s position that Auto Mode is a best-effort classifier and that OS isolation plus network-egress control are the real boundary. For unattended coding, use a container, VM, or OS sandbox, restrict egress, monitor the agent, withhold home directories and credentials, and never treat an Auto Mode approval as evidence that code is safe.
⚡ TRY THIS
Audit the harness, not just the prompt. Run
/doctorinside a Claude Code session every few weeks—claude doctorin a shell is only installation diagnostics—and inspect/memoryseparately. Addy Osmani’s audit covers unused skills, MCP servers and plugins relative to context cost, oversizedCLAUDE.mdfiles, slow hooks, and cruft. Then retry a representative task with local skills disabled and archive anything that does not earn its place; keepCLAUDE.md/AGENTS.mdfocused on checks, expensive commands, generated files, boundaries, and unusual conventions, while moving rules that must always hold into tests, hooks, or permissions.Use a behavioral model as a PR queue worker. Start with Theo’s prompt—“Take a look at all the pull requests I currently have open in this repo. Help me prioritize them based on ease of merge and the value that they provide to our users.”—then steer in place: exclude Codex PRs, keep the Fable/Cloud Code set, and request an HTML report for phone review. In the broader run, six subagents assessed overlap, fixed issues, merge risk, confidence, and closure; three hit provider/API failures, but the agent recovered by reading GitHub metadata, diffs, reviews, CI comments, and local source context, auditing hundreds of PRs in about 20 minutes and surfacing eight easy merges. Theo’s selection rule is useful: prioritize staying on task, accepting mid-run steering, and self-unblocking over raw intelligence; he later corrected the cost of a 1,000-plus-PR audit from roughly 50 cents to 12 cents.
Turn rewrites into phased, self-checking migrations. Start with easy or frequently updated files, add a retry loop, then build a more capable pipeline for the hard remainder: Airbnb moved 75% of 3,500 Enzyme test files in four hours, reached 97% after four days, and left the final 3% to engineers. For a product-scale conversion, feed the agent production runtime types, segmented tests, and an incremental boundary; Mike Krieger describes a dynamic workflow that ported and repeatedly verified a couple hundred thousand lines of Python to TypeScript over a weekend. Treat ROI headlines carefully: Asana’s roughly $6 million pre-AI baseline was a back-of-the-envelope estimate, not observed spend.
Separate personal context from write access. Peter Steinberger’s setup uses Telegram/OpenClaw for brainstorming and personal context, Codex for coding, and a context call from Codex to Claw only when needed. Start the execution agent read-only or restrict it to trusted folders; useful delegation does not require production access.
📡 WHAT SHIPPED
Anthropic’s Model Hardware Standard entered research preview. MHS is a model-agnostic specification for programmable lab and manufacturing equipment: standardized
read/writeprimitives and device discovery replace bespoke translators, while natural-language hardware tags generate reference files containing capabilities and enforced safety limits. Agents can operate devices through MCP, the CLI, or code files, then chain learned sequences into deterministic scripts for long-running work. This is still a preview rather than an open-source release; hardware needs a programming interface, and Anthropic says Claude Code’s physical reasoning still requires expert oversight.LangChain tightened the deploy loop. Managed Deep Agents can bake an environment once at deploy time from
setup.shor a Dockerfile, so new threads start without cloning or reinstalling. LangChain also says agents built in natural language can deploy to Slack in one click through its “Add to Slack” partnership.GLM 5.3 Flash is the Ox Alpha model Theo actually kept using. His field report describes a multimodal, one-million-token, open-weight model with hybrid attention and roughly $0.09 per task versus about $0.05 for Luna; the trade-off is token efficiency—about 47,000 tokens per task versus 20,000 for Luna Max and 17,000 for Soul. Theo’s conclusion is task-specific: keep Fable/Soul for hard problems, but use GLM for cheap, persistent, vision-capable agent work; during the free testing window he says it became the top model on OpenRouter and OpenCode.
OpenClaw is pushing toward code-native tool use. The project is fully open-source TypeScript, model-agnostic, and exposes a plugin interface for other model companies and harnesses. Its maintainers are experimenting on a branch with “code mode,” where the agent writes JavaScript for tool calls rather than issuing only regular calls—a proposed way to reduce loops and unify MCP invocation, not a finished feature.
T3 Code crossed 20,000 GitHub stars, while its instruction layer got better. Theo attributes a substantial PR-quality improvement to tuning
agentsmd/claudemd; he says the clearest gain was that PR names and descriptions became “100% easier to understand,” while pushing for simple, concise changes also improved the code.
🎬 GO DEEPER
- Theo — “Ox Alpha is INSANE”: Watch the PR-audit fallback and model-behavior sections. The useful lesson is not the leaderboard claim; it is the agent recovering from failed subagents, filtering a large PR portfolio, and producing merge candidates while Theo explains why obedience and persistence can beat raw intelligence for cheap agentic work.
- Mike Krieger — “How Anthropic Builds: Lessons from Labs”: Use the end-state-delegation and migration segments, then the discussion of async agents that own a slice of a codebase, monitor feedback, and remain human-driven at review time.
- OpenClaw maintainers — “OpenClaw Went Viral. Meet the Maintainers Building and Securing It.”: The practical section is the Codex↔Claw context bridge and the least-privilege rollout; the later discussion shows how the project replaces telemetry with agent-assisted crawling of Discord, Twitter, GitHub issues, and PRs to group problems and prioritize fixes.
Editorial take: The winning agent stack is becoming less trusting and more structured: sandbox execution, lean configuration, behavioral model routing, staged verification, and code-native tool calls matter more than another generic “best model” ranking.
The most useful highlight candidates are below. The supplied transcripts have numbered lines but no per-line clock locators, so exact video timestamps cannot be defensibly supplied; map each cited range to a 2–5 minute clip in the player.
Theo — GLM 5.3 Flash PR audit
- PR-audit setup and live scope steering — candidate:
L13–L18(clock timestamp unavailable). Theo wires the model into Codex/T3 Code, asks it to rank open PRs by merge ease and user value, sees a filesystem-MCP read failure followed by command-line fallback, then narrows the task to Fable/Cloud Code PRs and requests an HTML report. - Subagent fan-out, failure recovery, and direct fallback — candidate:
L20–L25(clock timestamp unavailable). The audit asks for subagents to assess a five-day PR cohort for overlap, fixed issues, merge risk, confidence, closure, and action rank; six workers run, three hit provider/API failures, and the agent recovers by auditing directly from GitHub metadata, diffs, reviews, CI comments, and local source context, completing the large audit in about 20 minutes. - Model-fit and cost reasoning for agentic work — candidate:
L65–L76(clock timestamp unavailable). Theo first gives a rough 50-cent estimate for the broad audit, then corrects it to 12 cents; he says GLM 5.3 Flash is weaker than Fable or Soul on hard problems but dramatically cheaper for easy tasks, while its vision and better task persistence make it preferable for some longer-running work.
Mike Krieger / Anthropic workflow interview
- From task delegation to end-state delegation — candidate:
L10–L13(clock timestamp unavailable). Krieger describes shifting from mentally decomposing an engineering task into steps toward stating the desired end state, letting the model work, and then discussing the trade-offs and questions it surfaces. - Whole-codebase conversion with verification and rollout controls — candidate:
L25–L33(clock timestamp unavailable). He describes using a dynamic workflow to port a Python codebase to TypeScript over a weekend, repeatedly verifying and double-checking the result; he connects this style of conversion with production-derived type data, segmented tests, incremental boundaries, pre-measurement, feature flags, ramp-outs, and dynamic runtime configuration. - Async proactive orchestration plus human-driven review — candidate:
L42–L53(clock timestamp unavailable). Anthropic uses interactive Claude Code for high-bandwidth work but delegates much of its usage through tagging, where an agent can own part of a codebase, monitor feedback, take tasks, make fixes, and react to API changes; important reviews remain human-driven, with artifacts explaining intent and trade-offs and Claude investigating the reviewer’s questions.
OpenClaw maintainers roundtable
- PR-volume controls and agent-generated contribution review — candidate:
L34–L50(clock timestamp unavailable). OpenClaw imposed a ten-open-PR cap per contributor to reduce noise; maintainers also encountered automated factories producing hundreds of PRs, duplicated work, and API-heavy attempts to claim credit, leading to a norm of directly editing promising PRs and using agent hints rather than forcing endless contributor back-and-forth. - Security boundary for initial deployment — candidate:
L71–L78(clock timestamp unavailable). Peter says the project hardened its codebase and channel integrations with outside partners, while recommending that cautious users start with a read-only agent or one restricted to selected folders; Brad emphasizes that secure organizational use still depends on identity controls, expertise, due diligence, and team governance. - Enterprise sandboxing, auditability, and capability trade-offs — candidate:
L83–L89(clock timestamp unavailable). The maintainers describe NemoClaw as adding sandboxing, a privacy router, and immutable audit logging; those controls slow and constrain the agent but provide visibility and accountability for regulated environments, illustrating the trade-off between a tightly restricted tool and an agent capable of taking real actions.
Recommendation: Include the analysis, but present it as a reported exploit demonstration—not as a universal attack-success-rate claim. The post explicitly says its rates come from small samples, improved as payloads were iterated with Codex, and are representative of a motivated attack but not comprehensive.
- Exploit entry and tool transition: A
WebFetchrequest receives HTTP 415, after which Claude independently switches tocurl; the root URL then redirects with HTTP 303 to a ZIP archive. The post identifies this as steering the model toward the attacker’s path by making it useful for completing the task, rather than directly instructing the model what to do. - Decoy binary and self-written decoder: The archive contains encoded records, a native
decoder-darwinbinary, andstruct.py. Claude refuses to execute the supplied binary and instead writes a Python replacement decoder, which it runs from inside the extracted archive. - Actual execution primitive: The replacement imports
base64from the extracted directory. Because Python places that directory on the module search path andbase64importsstruct, the archive’sstruct.pyshadows the standard-library module; its module-level expression executes during import while decoding still produces valid JSON. - Payload progression and impact: The poisoned module launches an isolated child that downloads and executes a remote stage; the reported stage downloads and launches a native test payload, establishes a controlled C2 callback, and opens Calculator. The detached process can outlive the Claude conversation, while Auto Mode sees only the short, benign-looking decoder several hops before the payload.
- Observed variability: Some runs were compromised before Claude recognized the booby-trapped archive; the post says Claude sometimes incorrectly reported that it had not executed the malicious instructions. In a few runs, Claude attempted cleanup after detection, but Auto Mode denied the cleanup command.
- Reported ASR, with exact qualification: The
python3 -I -cC2-chain variant succeeded in 3/5 runs (60%); theclaude -precon-and-Calculator variant also succeeded in 3/5 (60%); and theclaude -pvariant producing Calculator plus a write outside the workspace succeeded in 4/5 (80%). These are small-sample results, not a comprehensive or universal ASR measurement. - Benchmark comparison caveat: The cited 0.00% Opus 5 Auto Mode result came from a fixed evaluation of 72 scenarios run 10 times each, and the chain described here was not in that set; the post therefore says the 0.00% benchmark result and a working exploit can both be true.
- Concrete mitigations: Run unattended coding agents in a container, VM, or OS sandbox; restrict network egress; monitor agent activity; do not expose home directories, SSH keys, or cloud credentials; use explicit ask/deny rules for process creation and sensitive paths; and do not treat Auto Mode approval as evidence that code is safe.
- Exploit-specific defensive behaviors: The post reports successful refusals when Claude analyzed the archive statically, avoided running Python from the archive root, used
python3 -I, ran the replacement decoder from a safe parent directory, or recognized module shadowing before triggering it. - Boundary qualification: The source attributes to Anthropic the position that Auto Mode is a convenience feature backed by a best-effort classifier rather than a security guarantee, with OS isolation and network-egress control as the real boundary; it also states that Auto Mode may reduce risk relative to
--dangerously-skip-permissionsbut is not itself a security boundary.
Direct answer: Anthropic is opening a research preview—not a general release—of the Model Hardware Standard (MHS) to an initial group of scientific labs and advanced manufacturers. MHS is intended to become an open-source, model-agnostic standard for agents operating programmable physical devices, but the announcement says that open-source release comes later.
- What is new: MHS introduces a standardized hardware driver with simple
read/writeprimitives and a common discovery format, replacing bespoke device-to-device translator programs. - Integration and device knowledge: Natural-language tags capture hardware characteristics and generate a reference file describing what a device measures, what can be adjusted, and which safety limits will be enforced.
- Agent/developer interface: Agents can control hardware through MCP, a command-line interface, or code files/APIs; these mechanisms support multi-device orchestration from a single line of code. Agents can sequence instruments, monitor results, adjust parameters in real time, and chain driver commands into code files for long-running or high-speed operations that do not require online reasoning at every step.
- Coding-agent relevance: Anthropic’s concrete example is a coding loop: Claude explored laser alignment by changing a setting and observing camera results, then packaged what it learned as a deterministic script that performs alignment with one command. This makes reusable code generation—not just conversational control—a central part of the hardware workflow.
- Claude Code’s stated role: The announcement explicitly says many developers already use Claude Code with individual pieces of physical equipment, while the next MHS phase aims to cover more devices developers build on. It does not claim that Claude Code itself is the standard; MHS is model-agnostic and accessible to any agent harness through standard protocols such as MCP.
- Safety design and limits: Safety is partly encoded in device reference files through enforced limits, but Anthropic stresses that Claude’s physical and spatial reasoning limitations still require expert oversight; one example required researchers to recognize foaming as a physical failure rather than a software bug. The preview is also being used to build safety evaluations, strengthen protections and policy enforcement for physical-world misuse, and produce safe-deployment guidance for the eventual open-source release.
- Important boundary: MHS currently does not work with hardware lacking a programming interface, so broader coverage depends on manufacturers adding MHS drivers. Early support is being developed across agent/robotics and hardware platforms, including AWS Strands Robots, Automata LINQ, ScanImage, QIAsymphony Connect, Tecan Fluent, and robotics platforms.
- Agent-to-agent context bridge: An OpenClaw maintainer reports using Telegram to brainstorm, Codex for most coding, and having Codex call their Claw when it needs personal context; Claw answers back. This is a practical split architecture: keep personal context in a separate assistant and let the coding agent query it on demand.
- Inspect–repair–fallback loop: Peter Steinberger’s initial WhatsApp Relay prototype took about an hour; in a later interaction, the agent received a file with no extension, identified it as audio, converted its format, detected that local Whisper was unavailable, found an OpenAI key, sent the file to the server for transcription, and replied with the resulting text.
- Least-privilege rollout: Peter frames OpenClaw as a channel into a coding agent that can run sandboxed or in YOLO mode; he recommends starting with a read-only agent or limiting it to one or more trusted folders, and notes that agents can provide substantial value without touching production systems.
- AI-assisted PR review and dependency hygiene: An OpenClaw maintainer uses GitHub Copilot’s review action on AI-generated PRs to explain every touched file, its purpose, and how it changed, improving code clarity and helping the maintainer learn the codebase. After supply-chain concerns, the team inspected dependencies closely, reduced core dependencies by rebuilding components when practical, and established relationships with dependency maintainers.
- Repository-scale agent triage: Peter says OpenClaw has zero product telemetry, so the team crawls Discord, Twitter, and GitHub and uses agents to detect emerging issues and discussion trends. Scheduled Codex automations then report the main problems across the control UI and mobile/desktop apps, assign P0/P1/P2 priorities, group related issues and PRs, and help derive resolutions.
- Composable, model-agnostic tooling: OpenClaw is fully open source, written in TypeScript, supports any model, and exposes a plugin interface so different companies can integrate their own model or agent harness. NemoClaw adds sandboxing, a privacy router, and immutable audit logging for regulated environments, with the stated trade-off of slower, more locked-down operation.
- Emerging code-based tool calling: The team is experimenting with “code mode,” in which the agent writes JavaScript to perform tool calls rather than relying only on regular calls; they expect this could reduce agent loops and unify MCP invocation, but it remains an experimental branch.
- GLM 5.3 Flash / Ox Alpha (firsthand): Theo says the anonymous Ox Alpha model was later identified as GLM 5.3 Flash, which he integrated into T3 Code through an OpenRouter binding behind a proxy. When the filesystem MCP read failed, the model fell back to shell commands and continued the task.
- Replicable PR-triage loop: Theo started with the prompt, “Take a look at all the pull requests I currently have open in this repo. Help me prioritize them based on ease of merge and the value that they provide to our users.” He then narrowed scope explicitly to Fable-created PRs in Cloud Code, excluded Codex PRs, and requested an HTML report for easier phone review. In a broader run, he asked the agent to inspect PRs updated in the prior five days, use many subagents, identify overlap and fixed issues, assess merge risk and confidence, recommend closures, and rank actions. Six subagents launched; three failed on API/provider errors, but the agent recovered by auditing GitHub metadata, diffs, reviews, CI comments, and local source context directly. It covered hundreds of PRs in about 20 minutes and produced links to eight easy merges, all of which Theo merged.
- Low-cost automation: At the quoted rate of $0.075 per million input tokens and $0.25 per million output tokens, Theo asked the agent to inspect input, output, and cached-token usage. He initially estimated the 1,000-plus-open-PR audit at $0.50, then corrected the figure to $0.12; he planned to run the audit every three hours and get pinged when a PR appeared ready to merge.
- Model-selection heuristic: Theo separates model “intelligence” or knowledge from “behavior”: for agentic coding, behavior means following instructions, staying on task, accepting new steering mid-run, and unblocking itself. His practical split is to retain Fable and Soul for harder problems while using GLM 5.3 Flash for cheaper, easier agentic work where persistence and multimodality matter; he says GLM is less capable on difficult problems but more thorough and better behaved. He contrasts it with DeepSeek V4 Flash, which he found lacked vision and became distracted on longer-running tasks.
- Capabilities and caveat: GLM 5.3 Flash has a one-million-token context window and accepts image, audio, and video input. Its official description highlights hybrid sparse/linear attention for lower long-context serving costs, flat pricing as context grows, and open weights. Theo’s main drawback is token efficiency: he cites roughly 47,000 tokens per task versus 20,000 for Luna Max and 17,000 for Soul; he notes that excess tokens increase completion time and fill the agent’s context faster.
- When to add orchestration: Addy Osmani describes running parallel cloud tasks on production client applications with real users, authentication, payments, and subscriptions, alongside open-source projects, book companion sites, tools, and personal apps. He recommends starting with Claude Code or Codex in multiple sessions, using strong specifications, embedded verification, and constraints; add a custom software factory when you need repeatable runs, agent handoffs, duplicate-issue prevention, preserved evidence, or a way to pause production when human review falls behind. He published a runnable Factory, demo app, and workshop based on an 82-minute sample run; its verifiers caught real problems, while verification, retries, browser checks, and human review made tasks two to four times longer.
- Replicable issue-to-PR loop: His prompt instructs the agent to read the GitHub issue and repository instructions, implement only the acceptance criteria, avoid authentication/billing/migrations/existing test assertions, work on a branch with a reviewable diff, run
npm run lint,npm test, andnpm run build, stop if a required check cannot run, and open a draft PR documenting checks, remaining risks, and human decisions—never merge. For orchestration, triage issues intoready-to-implement,ready-to-spec,needs-info, orwait-to-implement; the label acts as queue and lock because sessions pick up only issues marked ready. - Verification must be staged and adversarial: Run fast linting and type checks early; run full tests, mutation testing, browser testing, and security checks near draft-PR creation or afterward, optimizing for signal-to-noise rather than the number of checks and retaining real tests instead of summaries. A green run can still violate intent if the agent changes the test or satisfies it with incorrect behavior, so untrusted GitHub issues or Slack messages should be treated as potentially adversarial and agent runs isolated with only the secrets each task needs. Osmani also recommends pairing run states such as
success,flawed,blocked, andmanualwith per-stage timing: in his sample, a no-rejection finder took 7 minutes while favorites took 56 minutes with two rejections and a human decision; a manual handoff is complete only when the human knows the next action. - Control comprehension debt, not just execution: Osmani warns that dozens, hundreds, or thousands of parallel agents do not increase the operator’s cognitive bandwidth; he personally encounters five or ten simultaneous projects, features, or sessions. For important work, ask the agent to persist its trajectory and lessons locally or in the repository so decisions and rationale survive session compaction, and increase autonomy only as verification builds trust for that project and risk level. Human judgment should remain upstream on problem choice, architecture, intent, and the quality bar, and at the final evidence-based shipping decision.
- Firsthand Claude Code hygiene workflow: Addy Osmani runs Claude Code’s in-session
/doctorevery few weeks and reviews memory separately./doctorchecks unused skills, MCP servers and plugins against their context cost, oversizedCLAUDE.mdfiles, slow hooks, and cruft;claude doctorin a shell is only installation diagnostics. Installed skills do not inject their full bodies into every prompt: names and descriptions load for discovery within a default 1% context-window listing budget, while the body loads on invocation. - Replicable audit loop: Treat installing a skill and keeping it permanently as separate decisions. Periodically evaluate skill descriptions, triggers, steps, examples, and security, using Anthropic’s Skill Creator evals/benchmark mode where appropriate. Archive or remove stale and experimental skills, then retry a task with local skills disabled to compare the raw model-and-harness baseline; encode rules that must always hold in tests, hooks, or permissions rather than prose.
- Keep repository context narrowly operational: Use
CLAUDE.md/AGENTS.mdas a short decision guide, not a growing knowledge base; document facts the model cannot easily infer, such as the right checks, expensive commands, generated files, architectural boundaries, and unusual conventions, while avoiding generic “write clean code” advice. In 288 runs across 17 real tasks, a cited study found no clear correctness improvement from context files, but a warning about a slow full test suite caused Claude to run more targeted tests and waste less time. A June study of 100 popular repositories reported lint-related leakage in 62%, context bloat in 42%, and skill leakage in 35%. - Contrarian personalization finding: A paper discussed by Osmani found that a skill derived from one developer’s interaction history performed about as well as a skill borrowed from another developer, while a generic skill built from many developers was more useful overall; personalization looked more promising only when the same preference recurred across similar tasks, and the experiments used an LLM-based developer simulator. Start with a strong generic skill, promote personal rules only after repeated evidence, and favor concrete task examples over one-off stylistic preferences. Osmani maintains the SDLC-focused Agent Skills pack, while Paul Bakaus maintains the design-focused Impeccable pack.
- End-state delegation (firsthand): Mike Krieger says he moved from decomposing work into conventional engineering steps to stating the desired end state, letting the coding agent work, and then reviewing the trade-offs and questions it surfaced. When the result is too opaque, he asks the agent to explain its decisions more simply.
- Large migration workflow: For a Python Labs project, Krieger created a dynamic workflow with Claude Code to port a codebase of roughly a couple hundred thousand lines to TypeScript for a better Bun deployment path. He had the agent port, verify, double-check, and repeatedly iterate over the weekend, returning to a completed, deployable result on Monday.
- Production-grounded incremental rewrites: Drawing on Instagram’s Monkey Type practice, Krieger recommends capturing runtime types from production and mapping them back into the codebase; for LLM-assisted language conversions, he suggests combining production data with segmented tests and finding an incremental migration boundary rather than attempting an overnight rewrite.
- Async proactive orchestration: Krieger says Anthropic uses interactive Claude Code for high-bandwidth iteration but delegates most work through a tagging workflow. The more advanced pattern is to assign the agent responsibility for part of a codebase, have it monitor a feedback channel, proactively take and fix tasks, and adapt to API changes—treating it as a teammate with context, memory, and initiative.
- Human-led, intent-focused review: Architecture work remains constrained by human review and the ability to understand what changed. Krieger’s team shares artifacts containing the change’s intent and trade-offs, and he asks Claude to investigate specific review questions instead of manually inspecting every line; lower-risk cosmetic changes can be fixed forward if needed.
- Dark Bloom is an early peer-to-peer inference network that serves open-weight models such as Qwen 3.6, Gemma 4, and GPT OSS; Matthew Berman reports that it is available through OpenRouter at roughly 50% lower cost than other providers and had served 4.5 billion tokens in about a week. This is relevant to coding-agent builders as a potentially lower-cost inference backend, although the video does not report coding-specific evaluations.
- Provider setup workflow: create a Dark Bloom account, install its CLI on a Mac, and run the single setup command; Berman used Codex with the prompt “install and set up Dark Bloom AI,” then signed in through Codex’s browser, entered the terminal code, approved the macOS Device Management enrollment, and ran hardware verification. The current minimum provider requirement is 48 GB RAM; his machine verified GPT OSS 20B at 78 tokens/second.
- Privacy architecture: Dark Bloom says inference runs inside one hardened Swift process with no subprocesses, local server, or interprocess communication, using MLX Swift LM on Apple silicon; this is intended to prevent the machine owner from seeing users’ prompts or responses.
- Caveat from firsthand testing: Berman had installed the software but had not yet begun serving requests or received payment, and characterized the project as experimental and in its earliest stage. He notes that the software is publicly readable and that an LLM review found no malware, mining, or credential theft, but current revenue-sharing terms can change and users should proceed at their own risk.
-
Anthropic’s Claude Code Opus 5 Auto Mode, recently made the default, is reported by Simon Willison to have been bypassed by prompt-injection researcher Johann Rehberger. Rehberger claims an attack succeeds 80% of the time: trick the agent into downloading and unpacking a ZIP archive, then executing code that imports
base64while a malicious localstruct.pyfrom the archive is imported and executed. - The failure can invert the safety mechanism: in some runs, Claude detected the compromise and tried to terminate the malware, but Auto Mode blocked the cleanup command; the classifier had allowed malware creation while denying the remediation step.
- For unattended coding agents exposed to adversarial inputs, the recommended baseline is to run inside a container, VM, or OS sandbox; restrict network egress; monitor the agent; and withhold home directories, SSH keys, and cloud credentials from the runtime.
- Hyper Agent workflow (sponsored segment): The video describes a multi-agent open-source maintenance loop in which one agent labels and reproduces an incoming bug, another investigates the fix and opens a draft pull request, and a human is contacted only for merge approval. The agents are shared across maintainers so project-specific knowledge is taught once; the segment also claims support for “DeepS” and a top open-weight model.
- Cross-provider agent orchestration: Kent C. Dodds says he repositioned Kodykoala to make it easier to use multiple agents from different companies, and says Kodykoala works similarly to the referenced
@bot. This is Kent’s firsthand product-positioning claim. - Credential-isolation pattern (secondhand example amplified by Kent): Store agent credentials in a vault and expose them through a host-controlled access path that never gives the model the secret in plaintext. For a request such as “file a bug that login is broken,” the model emits a Linear tool call containing only the title; the host identifies the tool, retrieves the agent’s Linear API key, constructs the real request, and returns the result.
For production AI agents, Addy Osmani highlights semantic caching to avoid spending tokens answering repeated questions, noting that agents can consume roughly 4× the tokens of chat. He points to Redis LangCache as a managed semantic-caching layer; Redis cites up to 90% lower API costs. The post is marked #ad, and the savings figure is presented as Redis’s claim rather than an independent benchmark.
- Poteto built an “island” interface that visualizes his bots as they work, rest, and return to their individual houses—an observable agent-activity UI pattern.
- Grok API integration: Poteto says Grok already has an API, describes it as limited but sufficient for his intended bot project. Kent C. Dodds reports accessing his Grok bots from an Android phone through Discord and shares a public bot interface at kody.codes/@kentcdodds/grok-bot.
@kentcdodds reports a firsthand mobile-access workflow: he can talk to his Grok bots from an Android phone through Discord. He says this setup is made easy by @kodykoala and @bot. A related resource is https://kody.codes/@kentcdodds/grok-bot.
Ben Tossell reports that his agent swapped Huge Icons for Lucide while working on an iOS project.
- Cursor announced a workflow for creating new web apps in Cursor, storing the code with Origin, and deploying to Vercel.
- Kent C. Dodds reacted, “About time,” but did not provide a firsthand workflow, benchmark, or tool comparison.
- T3 Code release: Theo highlighted a release with more than 380 changes, including in-app PR reviews,
npx t3 triagefor debugging and fixing T3 Code issues,npx t3 connectto turn a Mac into a remote development machine, one-click opening of remote projects in VS Code over SSH, and an overhauled in-app terminal. - Coding-agent workflow improvements: The release makes Codex skill discovery more reliable, exposes OpenCode skills in the app, enables Claude to discover project-level skills, surfaces Claude Code compaction recommendations, and surfaces Codex permission requests for MCP and Computer Use.
- Performance and operations claims: T3 Code reports a 93.8%+ improvement in rendering long threads, an 80%+ reduction in thread data transferred and stored, and remote-server upgrades that are 5× faster and 5× more reliable.
- Project traction and release channel: T3 Code 0.0.34 had 114 unique contributors, and nightly builds expose features that are mostly complete before stable releases.
- Claude + Model Hardware Standard (MHS): The demo team used MHS as a general interface between Claude and laboratory devices with different control languages, then defined explicit safe operating bounds; MHS refused a robotic-arm command that exceeded those bounds.
- Human-in-the-loop tool use: Because Claude had not seen the microscope application before and could make mistakes, the team treated operation as iterative: they monitored its actions, intervened when it blindly changed settings, and checked collision risks before higher-magnification moves.
- Make agent work observable: Claude generated an initial tracking script, but the team rejected invisible background execution and required a UI showing what it was doing; the resulting system tracked moving algae for several minutes.
- Closed-loop orchestration: In a liquid-handling experiment, the proposed pattern was: execute operations, inspect readings such as bubbles and transfer quality, adjust execution parameters, and rerun to improve the result.
- The team suggested this prototyping approach could reduce a PhD project’s setup period from two years to two months, but presented it as an intended outcome rather than a measured productivity benchmark.
GitHub highlighted a full OpenClaw maintainers roundtable featuring Peter Steinberger and other maintainers . Steinberger said the roundtable “turned out so good” and thanked GitHub . Resource: https://gh.io/openclaw-video
Do you need a software factory?
Do you need a software factory?

A software factory is a repeatable loop around software work. If you’re building a software factory, code good enough to ship still needs human taste and ownership. We’ll discuss this including whether you need a factory just yet.
If so:
You’ll likely need humans in the loop upfront for deciding on product intent, system design (if you care) and your quality bar.
Do review code (lights-on factory) but be intentional with where it’s needed the most. I’ve found you want to watch out for where automated back-pressure breaks. Or where maintainability trade-offs need to be made.
Aim for quality checks to happen as early and continuously as possible. Not all of them have to, but this includes type systems, automated tests, mutation testing, security scanners and linting for architecture rules.
Number of checks != quality. You’ll likely need to experiment with what checks give you the best signal to noise ratio. Be ready to tighten or relax your constraints deliberately.
You want to build your factory so some aspects of human taste get encoded in the environment, the agent gives you evidence of its work being right and where a human still “owns” what ships to production.
Do you really need a software factory?
In my experience, you can get surprisingly far with your stock coding harness! i.e. Claude Code or Codex, multiple sessions, good specs with verification baked in and constraints. You can even throw a batch of GitHub issues at them with implementation and human-involvement criteria. Claude Code routines and Copilot/Codex cloud agents already give you a scheduled or event-driven loop without custom infra.
A factory is worthwhile you need repeatable consistent runs, handing work off between agents, stopping two sessions from taking on the same issue, keeping evidence and pausing production when reviews are behind.

So I started off by saying a software factory is a repeatable loop around software work. We can actually look at a prompt that demonstrates a very small factory loop here:
Read GitHub issue #123 and the repository instructions before changing code.
Implement only the stated acceptance criteria. Do not modify authentication, billing, migrations, or existing test assertions. Work in a branch and keep the diff reviewable.
Run npm run lint, npm test, and npm run build. If a required check cannot run, stop and explain why. Open a draft pull request with the checks you ran, the remaining risks, and any decision a human still needs to make. Do not merge.
A goal can keep this moving until the checks pass and we can poll GitHub issues for any specific labels or review open pull requests each morning. Branch protection could enforce a merge boundary and the human can stay in the loop by choosing what becomes ready, reviewing and making the final merge calls etc.
Add a software factory when you need an event-driven queue of work (e.g. Slack triggers, GitHub issues, Linear, a backlog) to run in an isolated cloud environment to handle triage, implementation and testing with some explicit human babysitting. Some end their loop with a monitor agent watching production and filing issues which triage again.
In my experience, the factory becomes useful when the hard part is making your different runs behave consistently, handing work off between agents and avoiding different sessions from claiming the same issue, preserving evidence and stopping production when human review is falling behind.
What solves this might sound a little boring. For example, Warp mention triaging (opens in new tab) every incoming issue into one of four states - ready-to-implement, ready-to-spec, needs-info, wait-to-implement - and the label is what fires the next agent.

This label does a few jobs in one go: it’s the queue, the lock and since a session only picks up what’s marked ready, it’s where a human can park stuff without saying no permanently.
Workflow wise, there are a few similarities and differences to just using Claude/Codex:
Steering: agent needs course-correction, you can give input and redirect it
Notifications: how the factory says its blocked. This can be because a requirement was ambiguous, it started something risky or it needs human input (steering)
Handoff: move the task, its state and context between the cloud factory/another agent/human reviewer. Good handoffs will keep track of what happened, whats left to be done and why the handoff is needed.
In a good factory, the human isn’t limited to just reviewing and approving the final diff at the very end. They can shape the work early on, steer it during implementation, get it through a handoff or stop it shipping to production.
Verification is where a responsible factory spends a lot of its time. We’ll cover this more later.

If you decide you do need a software factory, building it isn’t the only option. Standing up the infra to scale a factory can be a lot of work and you may want to consider buying vs. building. Factory.ai (opens in new tab), Devin (opens in new tab), Warp (opens in new tab) and HumanLayer (opens in new tab) all sell some of this loop.
Where I actually spend my attention
My day-to-day experience of software development has changed a lot over the past year. I’ve been talking about increasingly doing a lot of parallel work with agents, moving towards having a lights-on software factory. And a lot of people have been asking me, like, what do these things actually mean? What are you building? What are the kinds of projects that you’re using these things on? It’s a lot of this:

So on a very average day, I have a simpler lights-on software factory. I can have tasks that are running in the cloud. Half of these tasks might be working on production client applications with smaller companies that I’m working with. They’re going to have real users. They’re going to have real authentication, payments, subscriptions, real beefy risks that you need to be careful with. You can’t just say, oh, agent, just go and do this stuff without having tests and constraints and quality checks in place.
I could work on my open source projects. I could be building out companion sites for my books. I could be working on tools. I could be building apps of my own. And these are all very, very different kinds of applications that I’m working on. And sometimes all they have in common is the tool that I’m using to work on them, right? Maybe I’m working on a migration. Others, I might be doing actual beefy feature work. And the blast radius of the work might also be very, very different.
So as you begin to think about getting to a place where we’re increasingly doing a lot of parallel work, we’re trying to improve velocity, we’re trying to improve productivity, and we’re trying to improve autonomy, which means getting the system to a place where we trust it more, you do have to think about what are the places that absolutely require human code review, human input.
And a lot of that is going to be required up front, right? When you’re defining your specification, your requirements, what is the design of the product going to look like? What is the intent of the product going to look like? And then how are you verifying that the agents have actually gotten the work done right? How are you verifying that they haven’t broken the existing system that has been in place? How are you making sure that it’s meeting your quality bar?
And so generating code is not necessarily the part that you need to worry about the most. Given enough context, agents can write the implementation and run the tests and inspect failure and revise code for us. We need to get to a place where we feel like there is enough of human taste encoded in the environment that we can trust what is being built, so that our human attention can be focused on the places where it’s needed most.
Now, there is pushback from folks saying, hey, well, I don’t buy that you can just automate away a lot of this stuff. It’s not to say that we’re automating away all of it, right? But given the volume of code that’s being generated, I don’t think that it’s realistic for humans to be reading all of it, especially when we’re not building rockets a lot of the time, right? We’re building UI, we’re building full stack applications.
Our judgment, our taste is best focused on the places where it’s needed the most. Like, what are the riskiest parts of the systems? Where do we need to apply human taste? And that can be in the front end. That can be in how the system works. It doesn’t have to be 100% of it.
My cognitive bandwidth does not scale with the agents
The reality is, yes, we can now fire up dozens, hundreds, thousands of agents in parallel, but your own cognitive bandwidth does not scale in the same way. This can feed into cognitive or comprehension debt which I’ve talked about before.

If you remember back to just five, ten years ago, there was a lot of discussion in the engineering community about context switching and the cost of it. We would talk about how people hated when a colleague or someone would walk up to your desk when you were in the middle of a task. It would then take you so long to get back into your flow state because you had to catch back up in terms of like, where was I? What was I doing? Even if you had a little bit of residue there, it still took you time.
We’re now context switching even more than we did before. On any given day, if I’m working outside of a software factory, I can be working on five or ten different projects with agents at a single time, or five or ten different features on a single project at a time. I can have five or ten different sessions, you can effectively say.
That means that I have to be able to stay on top of at least a few of those. It is possible that I’m going to be able to increase how much autonomy I give some tasks if I have trust that I’ve defined the task well enough, I’ve defined the outcome, how it’s going to verify that it’s done well enough. But then there are going to be tasks where maybe I don’t necessarily feel that way and there’s more risk involved or more nuance. I’m going to have to pay attention.
Consider optimising the software factory for your reviewer. Given every one of those approaches still routes its output to one person’s attention, you should ask how much cheaper the factory is making the decisions you still have to make.
A wrong-project mistake
I remember when I’ve been working on multiple parallel projects with my agents, and there have been times when I’ve accidentally done things like, maybe I was working on a web app where I wanted to add in a dark mode, and so I had in my head, okay, well, this is what the shape of this needs to look like. But I accidentally went to the session for a different project, and I started putting in that same prompt.
So I began implementing dark mode for something that absolutely didn’t need it. And so I can make that mistake. I don’t want my software factory making that kind of mistake.
You need to think about this really in terms of a system. You are effectively trying to encode a software engineering culture, a team culture, into a system so that it has those same kinds of behaviors, so that it has ownership that belongs somewhere, so that someone is still on the hook for what happens, and you’re being very explicit about how you think about those things.
When green is misleading
Even in these systems, you want to be very careful, right? Many of us have seen that when you have asked AI to help you pass a test, like we’re talking about a programming test, a unit test, it can change the unit test to satisfy that condition, or it can change the logic of the code to pass that condition. That doesn’t mean that it’s actually followed your intent in order to align both the functional behavior and what the test was supposed to be testing, right?

Just because a software factory is showing that everything is green doesn’t mean that it’s actually green, especially at the start when you’re setting these things up. You need to pay a lot of attention to make sure that your checks, your verifications, all of those are shaped the right way. They’re doing what you expect them to be doing. You don’t want them to be misleading.
You don’t want a situation where you had tests that said, hey, actually, I have gone and changed what authentication providers are supported. You asked me to add GitHub for example, as an authentication provider, but hey, my UI only had space for three, so I’ve gone and I’ve dropped one of the other ones. And hey, by the way, that happened to be one that your customers actually wanted. So you just need to be very explicit about how you want these systems to work.
Btw security is super important too and if your factory reads untrusted input like a GitHub issue/Slack message it might be adversarial and include problems like supply chain attacks. Some products around software factories, like Vercel (opens in new tab) Sandbox/AI ADK Factory, run their agents in isolated sandboxes holding just the secrets a task needs. That way a compromised run can’t reach what the job doesn’t need. Your defense ends up being layered.
Which old projects deserve another life?
I also think that a big part of how we work these days is deciding what should exist. If you remember back to many years ago before AI, there were so many abandoned software engineering projects, so many abandoned weekend projects, personal projects where they just wouldn’t launch because we didn’t have the time to finish them. We didn’t have the bandwidth to prioritize getting them out the door because they just weren’t that important to us or we couldn’t find the time.
Now it’s fairly trivial for us to complete those projects, but the same human judgment question comes in. Do those projects deserve to exist? Should they be launched? Because you put them out into the world and even if it has just five users, maybe you have to maintain it. Maybe you have a quality bar now that you want to maintain.
I know that I’ve had so many GitHub projects from over the years where now that I have an agent, the first thing I do is get the thing building. Because, of course, you clone it and now it doesn’t build because all the dependencies have changed. Half the things are out of date or have security vulnerabilities all over them, so you have to update that.
Then you have to add tests if you didn’t have tests so that you know that behavior is at least going to be there if you’re upgrading the project in some way, or if you’re migrating it to a more modern language or framework or thing like that.
Then you start to ask yourself, well, maybe, a silly example, but maybe I used Twitter Bootstrap back in the day for this, but now everybody is using Tailwind and shadcn, so I have to re-implement the UI. And what you’ll notice is that suddenly this is taking you more time, right? Yes, the agent can get a lot of this done quicker, but you’re now having to factor in product sense and taste and all of these things.
You still question, well, who is this for? Does it have a market? Is it for myself? Is it for other people? If I’m putting it out into the world, is it still going to be as interesting given that now anybody can spin these things up as quickly?
So I think that human question of do these things deserve to exist? How do we factor in our taste and judgment? I feel like those things continue to be extremely important. That’s where that scarce resource of human attention still really comes in. Back in the day, we only had a finite number of hours in the day. We had meetings. We had to budget in time for design and coding and so on.
Now that we have agents to help us, I think that you have to really just be very explicit about where you’re spending your time and why.
What happened when I built a sample one
So I’m going to talk about the 82-minute factory run. People have been asking me for quite some time, you know, “How do I build a software factory?” Or, “I’m used to using Claude Code or Codex. How do I evolve my setup to using a software factory?”
So the first thing that I’ve been saying is, “You may be fine. Your work may actually be totally fine without needing a factory.” But I did want to give people a reference setup that they can check out. So what I put together is a repository called Factory (opens in new tab) that you can go and check out. I also put together a demo application (opens in new tab) and workshop (opens in new tab).
Now, for the last couple of years, my go-to demo application for a lot of things has been a movies app. I’m a big movies fan. I love watching movies. I watch movies all the time, and so I have a demo application, which really starts off as a very simple movies app. And what I want the factory to be able to do is go ahead and implement a number of features. There’s a few different features. I want a favorites feature. I want it to be able to maybe do search, and maybe also want a dark theme in there as well, those types of things.
So I have my factory go and begin working with these things. You can check out the implementation. One of the benefits of it was actually catching real problems. These problems may not have been things that I would have caught if I had just asked it to do a one-shot implementation.
Maybe around the 60-minute point, I was feeling like, “Wow, this is going unusually slow.” I asked my harness using the factory, “Why are things going slow?” It said, “This is actually totally fine. All the verifiers are still running.”
You might have expected individual tasks to take 10 minutes, 15 minutes, 20 minutes, but they can take two to four times as long once you begin to include verification, retries, browser checks, human review, any of those extra delays.
I do think that these can add up to better quality and better trust in the system. From a measurement perspective, you might look at metrics like cost per merged PR and code shelf life as comprehension debt metrics.
You also need to think about what is useful delay versus factory overhead. The verifiers, in my case, caught some real problems. A little bit of the time was maybe sunk into producing evidence that I wanted. Some of it was overhead in the factory running. I didn’t really spend any time optimizing it, but a factory that just runs a lot of checks that you’re not finding valuable does not mean it’s a high quality one.
You want to study how, for any repeated checks, are they irrelevant? Are they noisy? Are they actually making the system safer?
Verification needs a budget
The way that I think about the budget for verification, this is basically what we’re talking about. We’re talking about a verification budget. I think about it in the same way as I’ve historically thought about performance budgets.

There are going to be certain kinds of checks that you can run early on in your software development lifecycle, and there are going to be some things that are so heavy, but they offer so much value that you will want to run them later on. There are some kinds of fast checks, linting, for example, type checking. These are relatively fast checks that you can run early on.
Our full suite of tests can be run closer to right before a draft PR is being put together or after that. That can include mutation testing, browser testing, security checks, anything like that.
I think that you don’t necessarily want to replace these with just summaries. You want real tests, but you just need to make sure that you’re budgeting for them in the right places because you don’t want to slow down your development loop. I certainly never want to slow down my development loop. Having a fast iteration loop is important to me, but I also want to still have those checks and balances.
When a run doesn’t ship
A lot of what I’ve written above concerns the checks.
In their software factory, Vercel marks (opens in new tab) every agent run as “success”, “flawed”, “blocked” or “manual” and only “success” ships to production. The rest re-enter the system. I’ve been thinking about runs in similar terms.

“Flawed” here means the wrong thing was implemented or maybe it didn’t have full context, so that has to be fixed. Blocked means the environment may have been missing a credential so you have to provide it. Manual is a boundary the factory may not be allowed to cross it yet.
Two of the three things here may have mechanical fixes and the last one is about trust.
While this is great what sorting doesn’t show you is cost. Back to my factory implementation with the TMDB app, the quick finder with no rejections took 7 minutes. Favorites, with two rejections and a human decision in the middle, took 56. Same factory. So I’d pair the taxonomy with per-stage timing, otherwise you know a run came back flawed without knowing what finding out cost you. The other thing I’d fix is the handoff at the boundary: my sample factory stopped the first issue and moved it to factory:needs-info, which was right, but I didn’t know where to put my answer. A manual run isn’t finished when the factory stops but when the human knows what to do next.
Autonomy is not a single setting
I wrote a couple of weeks ago an article about agentic autonomy (opens in new tab) and how to think about autonomy because autonomy is not going to be a single setting for every single project.
Verification buys you trust, and it buys the ability to grant more autonomy to your agents. So if, for example, I am working on a non-trivial change, but I have a number of checks in place, everything gets verified correctly, and maybe I’ve hand checked it myself. The next time I’m going to do a task like that in the same project, maybe I’ll feel comfortable giving the agent a little bit more autonomy.
That’s the thing that you think about when you’re building these software factories. Your verification is going to change with risk. Your goal is the best signal to noise ratio. You don’t just want to have some large checklist that you’re running.
The feature I had to relearn
There was a feature that I’ve been putting off on a day when I’ve been using multiple sessions with Claude, and I was working on a few different projects at a time, a few different features at a time per project. And so Claude had implemented the feature that I was working on. It looked like the tests were passing. I hadn’t put a lot of thought into verification, but the tests passed, and so I thought it worked. I merged it.
And so this was a favoriting feature. I thought that this was actually pretty good. I tried to check it out in the browser. It seemed like it was okay, but a couple of days later, I actually returned to the code because there were some tweaks that I thought I might make to this.
I didn’t want to just ask my agent to make the changes, because it was just a subtle way that it worked. You tap on the icon, and it would not show the right effect on tap, and so I wanted to just tweak it. I wanted to understand how it worked so I could guide my agent correctly.
I returned to the code, and I couldn’t explain to you how the feature worked. This repository was mine, right? I’d approved the change. I understood how a lot of it worked, a lot of the repo worked, but my understanding hadn’t kept up pace with all of the code that had been building up.
What I failed to absorb was how this feature that had been added actually worked, how the UI worked, how the effect on it worked. I had to redo this feature and actually go step-by-step, “How does this work? How can I understand it?”
What parallel work does to understanding
When you’re doing parallel work, it amplifies this overall problem, and it gets even more amplified when you’re doing it in a software factory. When you’re doing five or 10 sessions, they create much more than just a review volume problem. They create several mental models that can end up going pretty cold while you’re working elsewhere.
We’ve historically talked about the challenges with context switching, and as soon as chat compacts, you reject some approaches, you try out different things, you’re pairing with the agent, you’re going to have a difficult time remembering everything that happened in your session.
You can scroll up, and as compaction has been happening, you’re not going to have everything there, and you’re not going to be able to store it all in your head. Code often preserves a decision that was made, but not why the decision was made.
This is something that I think can be a useful learning for you, where it’s important, consider asking your agent to actually store information about its trajectory, or interesting lessons about how it approached a problem so that you can go back to it later.
This can or can’t be something that you decide to commit to a repo. You can keep it local if you want, you can share it with a team if you want, but that can be something that can then be consulted later on. Rather than you relying on it maybe being in a session, or you maybe remembering about it later.
What does it actually make?
@threepointone and @bentlegen posted some takes (opens in new tab) on (opens in new tab) software factories that I heavily agreed with:


It’s easy to get fascinated by the machinery of doing, optimize the machinery, and forget what it was supposed to produce. If your factory mostly produces a better factory, you’ve built software whose product is itself.
I think investing in the loop is fine and it compounds. The failure is when the loop closes, when everything the factory produces is consumed by the factory and nobody outside would notice if you switched it off. The test I’d apply is pull. Something outside has to be asking for the improvement. If you can’t name who’s pulling, you’re polishing.
Ownership doesn’t disappear
There is a broader principle underneath all of this.
The percentage of code physically typed by humans may fall dramatically. I don’t think human ownership needs to fall with it.
Someone still chooses the problem.
Someone still chooses the architecture.
Someone still sets the quality bar.
Someone decides which verification signals deserve trust.
Someone decides when the evidence is sufficient to ship.
And when the resulting system fails, “the agent wrote it” doesn’t cut it. This is why I don’t think the future of software engineering is best described as humans leaving the loop. Instead, human judgment is being relocated.
We should remove people from the parts of the loop where machines can produce stronger, faster, more deterministic signals. At the same time, we should concentrate people around the places where context, taste, risk, and long-term ownership matter most.
The best software factories will not be defined by how completely they eliminate human involvement.
They will be defined by how intelligently they place it.
Keep human judgment upstream on intent, system shape, and the quality bar. Review code where automated back-pressure becomes weak or the consequences become subjective. Push every deterministic signal as early and continuously into the loop as possible. Tighten and relax constraints deliberately as the system earns or loses trust.
A human still has to own what code ultimately ships. Code good enough to ship still starts with someone who cares whether it should exist.
- When to add orchestration: Addy Osmani describes running parallel cloud tasks on production client applications with real users, authentication, payments, and subscriptions, alongside open-source projects, book companion sites, tools, and personal apps. He recommends starting with Claude Code or Codex in multiple sessions, using strong specifications, embedded verification, and constraints; add a custom software factory when you need repeatable runs, agent handoffs, duplicate-issue prevention, preserved evidence, or a way to pause production when human review falls behind. He published a runnable Factory, demo app, and workshop based on an 82-minute sample run; its verifiers caught real problems, while verification, retries, browser checks, and human review made tasks two to four times longer.
- Replicable issue-to-PR loop: His prompt instructs the agent to read the GitHub issue and repository instructions, implement only the acceptance criteria, avoid authentication/billing/migrations/existing test assertions, work on a branch with a reviewable diff, run
npm run lint,npm test, andnpm run build, stop if a required check cannot run, and open a draft PR documenting checks, remaining risks, and human decisions—never merge. For orchestration, triage issues intoready-to-implement,ready-to-spec,needs-info, orwait-to-implement; the label acts as queue and lock because sessions pick up only issues marked ready. - Verification must be staged and adversarial: Run fast linting and type checks early; run full tests, mutation testing, browser testing, and security checks near draft-PR creation or afterward, optimizing for signal-to-noise rather than the number of checks and retaining real tests instead of summaries. A green run can still violate intent if the agent changes the test or satisfies it with incorrect behavior, so untrusted GitHub issues or Slack messages should be treated as potentially adversarial and agent runs isolated with only the secrets each task needs. Osmani also recommends pairing run states such as
success,flawed,blocked, andmanualwith per-stage timing: in his sample, a no-rejection finder took 7 minutes while favorites took 56 minutes with two rejections and a human decision; a manual handoff is complete only when the human knows the next action. - Control comprehension debt, not just execution: Osmani warns that dozens, hundreds, or thousands of parallel agents do not increase the operator’s cognitive bandwidth; he personally encounters five or ten simultaneous projects, features, or sessions. For important work, ask the agent to persist its trajectory and lessons locally or in the repository so decisions and rationale survive session compaction, and increase autonomy only as verification builds trust for that project and risk level. Human judgment should remain upstream on problem choice, architecture, intent, and the quality bar, and at the final evidence-based shipping decision.