🔥 TOP SIGNAL

Jev is becoming a credible control-plane component for agent evaluation, not just another model to compare. LangChain’s Daniel Shea and Seán Roche captured five weather-agent runs, replayed the identical traces across judges, and used human labels as the oracle. Jev matched the oracle on all 500 repeated binary decisions; its observed score variance was 92–913× lower than the other judges, and it averaged 0.44 seconds and $0.00035 per call in the test. @LangChain

Treat this as a strong harness signal, not a universal model verdict: the authors call the experiment early and narrow, and explicitly warn that a cheap evaluator can amplify systematic mistakes. Keep human review and judge-alignment checks in the loop. @LangChain

⚡ TRY THIS

• Put a typed judge in every regression loop. Capture fixed agent traces in LangSmith, replay the same traces across evaluator versions, and ask several atomic questions over the same state in one call. Use Choice for bounded judgments such as “is the answer grounded?”, Score for rubric-based usefulness, and Noul for yes/no-style decisions; track both agreement with a human oracle and repeatability, not variance alone. @LangChain

• Turn external events into typed work queues. Kent C. Dodds is triggering bots from email, Sentry, feedback submissions, package errors, and Discord; webhooks are the entry point, and the bot can explain how to configure them. Run the payload through a single decision pass for actionability, urgency, escalation, or “ready to ship,” then invoke a documented utility rather than asking an agent to improvise the final operation. Kody’s model is explicitly a skill plus a tool, intended to produce more reliable, repeatable outcomes; its merge-PR utility is the concrete example. @kentcdodds +3

• Separate model switching from terminal-session management. Geoffrey Huntley’s current hands-on recommendation is OpenCode as one harness for different model options and easy switching, paired with Herdr as an agent-aware alternative to tmux. That directly answers the recurring Codex-versus-Claude problem, but keep the build/test boundary explicit rather than assuming a cloud coordinator can run local work. @GeoffreyHuntley +1

• Test instruction files per model before standardizing them. John Lindquist wants a benchmark for the influence of AGENTS.md on each model; Huntley argues the convention was meant to be model-specific and proposes an x-user-agent signal so MCP servers can select model-specific system and tool prompts. Run the same coding task with a shared file versus model-specific variants, and log behavior, tool calls, and failures before declaring one instruction file universal. @johnlindquist +2

📡 WHAT SHIPPED

• Pi 0.86.0: adds mid-conversation system messages, dynamic tools on supported models without losing the KV cache, Anthropic cache warming, faster -r/-c resume, fixes, and a /bug command. Armin Ronacher warns that the system-message change may regress behavior because relatively few users run it from main; treat this as a canary upgrade, not a frictionless one. @pidotdev +1

• Claude Projects’ boundary is now clearer in practice. Riley Brown likes bundling sessions into projects with project-specific routines, but reports that the orchestrator cannot start a local Claude Code thread: every session runs in the cloud. That is convenient on iOS and awkward for app building, so use Projects for coordination only if the execution boundary fits your workflow. @rileybrown

• Bespoke Nimble is an open decision-model project worth watching. The release includes open data, code, and a 9B model, with a LoRA fine-tune of Qwen3.5-9B, synthetic contrastive data, and Jev used for evaluation rather than distillation. The author reports 66% for base Qwen versus 90% for Nimble on a curated eval and 100 ms on an H100, but also says there is no standard benchmark and Nimble may perform much worse elsewhere. A current follow-up says it counted bedrooms and bathrooms in a confusing floor-plan image where base Qwen struggled; Jev was not tested because it lacks multimodal support. @madiator +1

🎬 GO DEEPER

• Repo — jev-as-a-judge: Study the fixed-trace replay, human-oracle, repeatability, and cost methodology rather than copying the headline number. The article lists the package versions used for reproduction. @LangChain

• Repo — Bespoke Nimble: Read the contrastive-data recipe, then look for the missing ablation and out-of-domain benchmark before treating the reported lift as general. @madiator

• Video — Geoffrey Huntley’s AGENTS.md discussion: Pair it with his follow-up proposal for model identity in MCP; the useful question is whether instructions and tool prompts should vary with the active model. @GeoffreyHuntley +1

Editorial take: The practical frontier is a separated control plane—typed decisions, event triggers, model switching, and repeatable evals—wrapped around a coding agent whose local or cloud execution boundary is explicit. @LangChain +2