ZeroNoise Logo zeronoise
Post
Preflight Adds a Sanitizing Boundary to the Multi-Model Agent Stack
4 min read
70 docs
A new gateway pattern puts redaction, local attachment inspection, and model routing between coding agents and their providers; Jev’s latest practitioner guidance clarifies the narrow role of fast typed decisions.

🔥 TOP SIGNAL

Geoffrey Huntley’s preflight makes the agent gateway—not a prompt—the security boundary. His chain is client → preflight → underclass → model: Underclass collapses 20 ChatGPT/Copilot subscriptions into one endpoint, while Preflight listens on :8081 and starts with nix run github:ghuntley/preflight -- serve. It defaults to redaction rather than abortingghp_… becomes [REDACTED:github-pat]—while screenshots, PDFs, barcodes, and OCR are decoded locally; unsanitized attachments do not go upstream, and logs/HTTP 409s expose only finding and rule IDs.

⚡ TRY THIS

  • Put a security hop in front of the harness. Point the agent from :8080 to :8081, run nix run github:ghuntley/preflight -- serve, and keep the client → preflight → underclass → model path. Make repeated multimodal checks reusable by caching the verdict for the same attachment plus inspection profile content-addressably on disk for a one-week TTL; vendor and pin Gitleaks, update it through a CI-checked PR, and keep new rules disabled until explicitly allowed in defaults.toml.

  • Use Jev as a typed fast lane, not an agent. Pass structured state and a schema into it, use its confidence to set an automation threshold, and send below-threshold cases to review; keep codebase traversal, tool outputs, and history compaction in the reasoning model. Theo describes Jev as a function-like classifier with a 32k-token context and explicitly rejects using it to judge several LLM outputs or compact agent history; Salvatore Sanfilippo gives the confidence-to-human-review pattern.

  • Provision keys out of band on remote agent machines. Simon Willison’s llm-keys-ui workflow is: have Codex run uvx --with llm-keys-ui llm keys-ui --all, open the returned local-network or Tailscale URL to save the key, then let a later shell command retrieve it with llm keys get anthropic instead of pasting the secret into the agent session.

  • Use a project as the orchestration folder. In Riley Brown’s Claude Project pattern, name the project, give it a goal, let the main chat create task-specific threads, and have those threads—not the main agent—use tools. Assign models per thread, store their artifacts in the project library, and keep recurring routines inside the project so new threads inherit the project context without re-explaining the work.

📡 WHAT SHIPPED

  • llm-keys-ui 0.1: Simon Willison released the web UI for setting LLM keys, motivated by controlling Codex Remote sessions from a phone without pasting API keys into the agent or ChatGPT app.

  • Agent gateway pair — underclass and preflight: Underclass is Huntley’s one-endpoint consolidation of 20 ChatGPT/Copilot subscriptions; Preflight adds the redacting, attachment-aware checkpoint in front of it.

  • OpenClaw FaceTime support: Omar Shahine says the experimental plugin supports audio, lets users call their claw and the claw call them, supports GPT-live, and should arrive in the next release; Peter Steinberger separately announced that a claw can now FaceTime, so treat the rollout status as qualified.

  • Jev early access: Theo’s walkthrough positions Typesafe AI’s model as structured state in, typed probabilistic decisions out, available through OpenRouter and Vercel AI Gateway. He cites 70–500 ms classification and roughly $0.04 per million tokens versus $10 for Fable, but stresses the 32k-token context and Jev’s unsuitability for codebase traversal, LLM judging, or context compaction.

  • Kody’s shared capability layer: Kent C. Dodds is positioning Kody as a common place for multiple assistants to run integrations, secrets, automations, jobs, webhooks, and software packages; he describes the package feature as a personal software registry plus execution environment. In a separate firsthand example, his Grok bot used Kody’s calendar and email connections to manage a trip and draft a reply.

🎬 GO DEEPER

  • Video — Theo’s Jev is incredible: Watch the model-boundary segment first: Jev is a function-like classifier for structured state, not a code-writing model. The later section is the useful anti-pattern review—why judging multiple LLM outputs or compacting agent history throws away the context and tool results those tasks require.
  • Repo — preflight + underclass: Study the proxy boundary, default-redaction behavior, local multimodal inspection, content-addressed verdict cache, and CI-controlled Gitleaks rule updates before copying the pattern.

  • Repo — llm-keys-ui 0.1: A small, immediately reusable pattern for keeping API-key entry in a separate web surface while remote coding agents retrieve keys only at command execution time.

Editorial take: The practical frontier is boundary design: sanitize model calls, keep narrow decisions typed, and reserve human time to inspect generated work instead of optimizing only for “press enter.”

Preflight Adds a Sanitizing Boundary to the Multi-Model Agent Stack
Theo - t3․gg
  • Jev’s role and interface: Theo describes Typesafe AI’s Jev as a “system 1” classifier for fast structured decisions rather than text or code generation; it is intended to be called from application code like a function, with structured data in and typed JSON out. Theo says it was invite-only early access at the time, with availability through OpenRouter and Vercel AI Gateway. Its output shape is deterministic and it reports confidence/uncertainty, but the semantic answer itself is not deterministic; its context window is only 32k tokens.
  • Recommended agent architecture: Keep a reasoning model for codebase traversal, tool use, and multi-step decisions, while using Jev as a fast typed classifier, router, ranker, or guardrail over simple structured state and batch data. Theo explicitly rejects using Jev to judge several LLM outputs or to replace agent context compaction: it lacks sufficient context, tool-call results, and reasoning traces, while compaction synthesizes the full history rather than merely filtering lines. He recommends relying on the existing harnesses’ compaction behavior instead of reinventing it with a shallow classifier.
  • Structured-output workflow: Theo’s BAML example defines a positive | negative | neutral label, a verdict containing label and confidence: float, and a classify(text) function callable from TypeScript. He reports using BAML with GPT OSS 120B and finding that the model formatted JSON incorrectly about half the time without BAML; BAML’s runtime repairs malformed output to match the declared schema. The practical constraint is that a valid schema does not guarantee a correct decision, so confidence should remain part of downstream routing rather than treating typed output as proof of correctness.
  • Performance and firsthand batch use: Theo cites Jev at roughly 4 cents per million tokens versus $10 per million for Fable, and 70–500 ms for classification versus 3 to more than 300 seconds for traditional LMs. In his own test, eight workers processed 100 emails at an average of 200 ms each, 240 ms p95, and 38 per second; he positioned this as a cheap first-pass classifier for work that is too frequent or inexpensive to justify a general LLM. He also classified 32,311 messages across 1,118 T3 code threads, using confidence thresholds: “expanding scope” appeared in 22% of threads at an 80% threshold but only 6.8% at 90%.
  • State-management caveat for agent loops: In a Doom demo, calling Jev on every frame without prior action state caused it to alternate left and right because each decision effectively started fresh. For repeated control loops, carry the relevant prior state into the structured input or use the model for bounded individual decisions rather than expecting it to maintain a long-running plan.
Jev is incredible
Salvatore Sanfilippo
Profile
  • Model routing and confidence gating: Salvatore Sanfilippo describes “Jev” as a small, non-autoregressive language model designed for fast, inexpensive classification with calibrated label probabilities. He contrasts it with post-trained autoregressive LLMs, which can handle harder cases through reasoning and structured JSON output but whose teacher-forced token probabilities may be poorly calibrated.
  • Human-in-the-loop pattern: For automated classification, set a confidence threshold and send lower-confidence cases for human review; the example is to accept an answer only above a stated “0,85%” threshold and otherwise route the email for scrutiny. The specialized classifier is therefore best treated as a narrow, low-cost routing component rather than a general-purpose agent replacement.
Chi se ne frega di Jev?
ThePrimeagen
  • ThePrimeagen warns that blindly pursuing AI-generated features without understanding how they work—“feature lust”—will end badly; greater model intelligence is not a substitute for human judgment.
  • In a quoted @v0xium account from a new role at a large company, Claude Code was reportedly generating specs, code, tests, PRDs, tickets, ticket resolutions, and reports. Pressure to ship led people to work 12–13 hours per day while “nobody is reading anything” or resolving bugs. The practical safeguard is to reserve explicit human time to inspect generated code and trace its behavior before shipping, rather than optimizing only for output volume.
I hate this. I hate that people, even when slopping, don't even think about how it works. This has to end poorly and I don't see how an i… I am done with this shit. It is over. The state of engineering right now is horrible. It has been half a month since I started a new role…
Brent Traut
  • Human-in-the-loop multimodal workflow (firsthand, cross-domain): Brent Traut reports that ChatGPT used photos to identify the products/models, determine a best price, complete the Marketplace forms, and ask for confirmation only on the final step; he says the sale generated $400 in 20 minutes. For coding-agent design, the transferable pattern is to let the agent execute the multi-step research-and-entry work, then place a human approval gate immediately before the consequential external action.
Just sold some junk on Marketplace. I took a few pictures, then ChatGPT found 'em, figured out what the products/models were, figured out…
Simon Willison's Weblog
  • Simon Willison (firsthand workflow): He uses Codex Remote to run coding agents on other machines while controlling them from his phone, and built llm-keys-ui 0.1 to provision API keys without pasting them into the agent or ChatGPT session. Run uvx --with llm-keys-ui llm keys-ui --all in Codex, have the agent return a URL reachable over the local network or Tailscale, save the key through the web interface, and later retrieve it inside a shell command with llm keys get anthropic (or another provider name).
llm-keys-ui 0.1
Peter Steinberger 🦞
  • OpenClaw is adding experimental FaceTime support, with audio for now: users can call their claw, the claw can call users, and the feature supports GPT-live. Omar Shahine said it was expected in the next release and linked the plugin documentation: https://docs.openclaw.ai/plugins/facetime/
  • In Shahine’s firsthand development account, FaceTime was the longest OpenClaw feature he had worked on; he said Opus 5 and GPT-Sol 5.6 were the models that enabled him to finish it.
  • Peter Steinberger separately announced that “Your claw can now FaceTime you,” while Shahine’s linked details describe the feature as coming soon and experimental, leaving the rollout status qualified.
Give your Claw FaceTime (coming soon). This was the longest I've worked on any [@openclaw](https://x.com/openclaw) feature. FaceTime supp… Your claw can now FaceTime you! [https://x.com/OmarShahine/status/2101727325947863430](https://x.com/OmarShahine/status/2101727325947863430)
Simon Willison's Weblog
  • Contrarian warning from voxium: In a new role at a large company, voxium reports that Claude Code produces the specs, code, tests, PRDs, tickets, ticket resolutions, and reports, while engineers from L1 through L7 allegedly spend 12–13 hours a day “just to press enter” and “nobody is reading anything.” Management reportedly treats code production as non-bottleneck, highlighting the risk that maximizing agent output without mandatory human review can create a severe comprehension and quality-control problem.
Quoting voxium
geoff
  • @GeoffreyHuntley describes preflight as a checkpoint before the model because coding agents may paste .env contents into it. The proposed path is client → preflight → underclass → model; Underclass consolidates 20 subscriptions behind one endpoint. To adopt it, point the harness at :8081 instead of :8080 and run nix run github:ghuntley/preflight -- serve; the referenced repo is https://github.com/ghuntley/preflight.
  • The gateway defaults to redaction rather than aborting: for example, a GitHub token becomes [REDACTED:github-pat] and the request continues. Screenshots, PDFs, barcodes, and OCR are decoded locally; unsanitized attachments are not sent upstream, while logs and HTTP 409 responses expose only a finding ID and rule ID rather than the secret.
  • To avoid repeated multimodal work, cache verdicts content-addressably on disk by attachment and inspection profile with a one-week TTL; Huntley says the first screenshot inspection incurs the cost and the next 20 reuses are free. For secret detection, he uses vendored, pinned Gitleaks rules; a weekly job pulls the latest stable snapshot, opens a PR, and squash-merges only when CI is green, while new upstream rules remain inactive until explicitly enabled through defaults.toml.
These agent's can't be trusted to not paste your .env into the model. So I put a checkpoint in front of it. underclass made 20 subscripti… The default is redact, not a lecture. ghp\_… becomes [REDACTED:github-pat] the request keeps going the agent does not die because it cat'… text is the easy case. screenshots, PDFs, barcodes, OCR are decoded on the box. if it cannot sanitize the attachment, the request stays g… OCR is expensive. doing it twice is stupid. same attachment + same inspection profile → reuse the verdict. content-addressed. on disk. TT… rules are Gitleaks. vendored. pinned. a weekly job pulls the latest stable snapshot, opens a PR, squash-merges when CI is green. new upst…
Kent C. Dodds 🐨

The author reports a firsthand workflow in which the Grok bot “Scott the Sloth” used Kody connections to access his calendar and email, manage an entire trip, and draft a reply after determining that his schedule had no room; the attached draft concerns speaking at Nerdearla.

Scott the Sloth is one of my Grok bots and it gets an email thanks to [@kodykoala](https://x.com/kodykoala). Scott has managed the entire…
Theo - t3.gg
  • Theo argues that, for real-world code work, it is difficult to justify using models less capable than Fable and Astra because their benefits “massively outweigh the cost”; he says the advantage is not better code but a subtler benefit that he does not explain here.
  • Theo says the concept of “prompt wider” is resonating with users, but this post does not define the technique or provide steps for applying it.
It's hard to justify using models that are dumber than Fable and Astra. For real-world code work, the benefits massively outweigh the cos… Been getting a lot of positive feedback on this one. Thank you guys! I'm pumped the concept of "prompt wider" is resonating with y'all. 🫡…
Kent C. Dodds 🐨
  • Multi-assistant workspace: Kent C. Dodds recommends kody.codes for people using more than one assistant, describing it as a shared place to put and run work with integrations, secrets, automations, jobs, webhooks, and software packages. The linked example from Nate Eliason separates assistants by role: Muse for personal assistance and GrokBot for work assistance.
If you're using more than one assistant, may I recommend [https://kody.codes](https://kody.codes) to give them a common place to put and … Muse -> personal assistant GrokBot -> work assistant All you need now [https://x.com/nateliason/status/2101761903437496491](https:/…
Kent C. Dodds 🐨
  • Kody is described as having a feature that functions as a personal software registry and execution environment, similar to an npm package, so connected agents can run the registered software.
This is one of Kody's best features, but it takes a bit of explaining... Sigh... Still working on it! If your mind went to "oh, like an n…
geoff
  • Geoffrey Huntley reports building underclass to consolidate 20 ChatGPT/Copilot subscriptions—rather than treating them as separate accounts—behind a single endpoint. This is a firsthand tool announcement; the repository is github.com/ghuntley/underclass.
I got tired of treating 20 ChatGPT/Copilot subscriptions like 20 separate accounts. So I made them one endpoint. [https://github.com/ghun…
Kent C. Dodds 🐨

@devagrawal09 argues that “deterministic instructions” are better than Markdown instructions for guiding coding agents. @kentcdodds points readers who agree to try Kody, but provides no workflow, configuration, benchmark, or firsthand usage details.

[@kentcdodds](https://x.com/kentcdodds) YES deterministic instructions >> markdown instructions If you agree, try [https://kody.codes](https://kody.codes) [https://x.com/devagrawal09/status/2101626484574363663](https://x.com/devagraw…
Riley Brown
  • Riley Brown describes a Claude Project as a “Folder For Organized Agent Orchestration” (FFOAO): name the project, define its goal, and use a main chat that can spin up task-specific threads.
  • In this setup, threads—not the main agent—use tools; each thread can use a different model, create editable artifacts, and inherit context from the entire project so tasks do not need to be re-explained.
  • Keep recurring routines/cron jobs inside the project and store generated artifacts in the project’s library.
I'm "claude project" pilled. A Claude Project can be thought of as a "FFOAO" A Folder For Organized Agent Orchestration And it's very int…
Theo - t3.gg

Theo called Jev “incredible” but warned that people may be using it incorrectly.

Jev is incredible. I'm scared people are using it wrong. [![Video](https://pbs.twimg.com/amplify_video_thumb/2101856554370891776/img/uDkc…