ZeroNoise Logo zeronoise
Post
Jalapeño Pushes AI Competition Into the Inference Stack
1 day ago
4 min read
830 docs
OpenAI’s Jalapeño puts inference silicon at the center of AI competition, while local-first agents, robot-data infrastructure, harness research, and export-control enforcement show the stack broadening around the model.

Top Stories

Why it matters: AI competition is moving below the model layer—to inference silicon and deployment architecture.

OpenAI’s Jalapeño makes the serving stack strategic. OpenAI reports 1.5–1.9× more AI work per watt and 1.7–3.6× lower end-to-end latency across GPT‑OSS 120B, DeepSeek R1, and Kimi K2.5 1T in InferenceX, with 2.1–4.1× higher performance on highly interactive workloads. It plans to deploy the chip by year-end, with Gen 2 in development and Gen 3 taking shape. OpenAI also says AI helped move Jalapeño from initial design to tapeout in nine months. The public setup uses nominal 8K/1K input/output and package-TDP comparisons, so this is a strong vendor-reported signal rather than independent proof.

Portable Computer makes “local” hybrid. Perplexity launched a DGX Spark version in which the orchestrator, subagent, and harness run locally with no cloud dependency. It reports 82.6% on real knowledge work for an on-device 27B model and 85.4% for post-trained PPLX 27B. For frontier reasoning, an approval-gated cloud fallback lifts Terminal Bench 2.1 from 59.6% to 73.0% at $0.415 per rollout. The privacy-by-default pattern is notable, but the test used a $5,000 dedicated AI computer—not an ordinary PC.

Research & Innovation

Why it matters: Agent reliability is becoming an engineering and evaluation discipline, not just a model attribute.

Harness choice can outweigh model choice. A controlled study across three models, three harnesses, and 100 SWE-bench Verified tasks found that swapping the harness moved GLM-5.1 by 13 points versus 2.5–5 points for model swaps; harness variance was 7.8× larger and six of nine model rankings flipped. It proposes a seven-layer Harness Card to separate model, harness, and interaction effects. AutoSaddler applies the same insight operationally: its offline loop patches prompts, tool configurations, and control logic from failure traces, reporting gains of 9.0, 9.6, and 10.0 points on GAIA2, SWE-Bench Pro, and Terminal-Bench 2.0.

Reasoning training can amplify the wrong behavior. Behavioral Lift analyzed 15,282 traces from 15 models across six benchmarks and found that thinking models strongly amplify self-correction, hypothesis testing, and uncertainty acknowledgment, while confidence calibration, knowledge alignment, and self-awareness have the highest correctness lift. Uncertainty acknowledgment was amplified 3–7× yet was weakly or negatively associated with correctness.

Products & Launches

Why it matters: Product differentiation is shifting toward agent-ready interfaces and inexpensive open models.

WebMCP is an experimental open standard for letting web apps expose tools directly to agents; ChatGPT’s desktop browser and Sites can automatically use compatible websites.

Qwen3.8-27B ranks first among open models and seventh overall in Arena’s Image-to-WebDev benchmark, at $0.40/$3 per million input/output tokens. Arena reports 1,574 points and performance on par with 2.8T-parameter Kimi K3 Max.

Industry Moves

Why it matters: Commercial AI scale is appearing in operational fleets and proprietary data infrastructure, not only model releases.

Figure’s Index reports 16M video uploads, more than 30 minutes of video per second, $15M paid out, 43,000 weekly active users, and collection in 108 countries. Figure has committed more than $1B over the next 12 months for data and compute, arguing that general-purpose robot training data must come from the real world.

Gatik raised a $200M Series D at a $1B valuation, led by Qatar Investment Authority and Koch Disruptive Technologies. It says driverless freight is already commercial, with more than 100,000 deliveries, 99%+ on-time performance, and over $600M in contracted revenue.

Policy & Regulation

Why it matters: Export controls are reaching the physical logistics chain that supplies AI compute.

A post citing Reuters reports that Taiwan prosecutors indicted nine people, including an Nvidia Taiwan employee and two former Super Micro Taiwan employees, over an alleged B300-server smuggling scheme. It says false documents claimed 130 servers would remain in Taiwan; 74 allegedly reached Chinese customers and customs intercepted 56.

Quick Takes

Why it matters: Evaluation integrity, agent access, and infrastructure throughput are advancing in parallel.

  • ChatGPT Work can now sign into websites without ChatGPT seeing the user’s username or password.
  • Artificial Analysis now assigns zero to Terminal-Bench attempts that fetch published solutions online.
  • vLLM and SkyRL report transferring Kimi K2’s 1T BF16 weights in 7.53 seconds across 48 nodes equipped with eight H100s each.
  • Alibaba previewed Qwen3.8-Flash-Next, an open-weight multimodal MoE built on the Qwen4 architecture, for release the following day.
Jalapeño Pushes AI Competition Into the Inference Stack
Research extraction

The supplied abstract verifies a study spanning 15 models and 6 benchmarks across text-only and vision-language reasoning, with 15,282 annotated reasoning traces; its taxonomy includes core behaviors defined for both LLM and VLM traces. The extract does not identify the six benchmarks or provide further dataset composition details.

The central finding is an “Amplification-Lift Gap”: thinking models strongly amplify self-correction, hypothesis testing, and uncertainty acknowledgment, while the behaviors with the highest correctness lift are confidence calibration, knowledge alignment, and self-awareness.

Confidence calibration is among the strongest positive correctness signals in both modalities but is barely amplified; uncertainty acknowledgment is amplified 3–7× yet is weakly or negatively associated with correctness.

The implication for reinforcement-learned reasoning behavior is that reasoning-oriented training can make traces appear more deliberative without preferentially amplifying behaviors most tied to correctness. The paper therefore motivates process-level objectives that reward calibrated and grounded reasoning rather than surface form alone.

Amplified Does Not Mean Predictive: Reasoning Behaviors in Thinking Models
Research extraction

The supplied abstract verifies the core AutoSaddler method, its three evaluation benchmarks, and the headline improvements, but it does not provide implementation-level evaluation details or explicit limitations. AutoSaddler frames harness improvement as an offline learning problem: it uses failure signals from mini-batches to iteratively update the harness, combining failure-trace diagnosis, structured patches that treat the harness as code, and validation-based update selection.

  • Benchmarks: GAIA2, SWE-Bench Pro, and Terminal-Bench 2.0.
  • Reported gains over the corresponding base harnesses: +9.0, +9.6, and +10.0 percentage points, respectively, in the benchmark order listed.
  • Ablations identify three claimed ingredients: deep debugging rather than shallow reflection; targeted modifications rather than unconstrained editing; and generalization-aware selection rather than trajectory-specific repair.
  • Limitation/gap: the supplied material is an abstract-level record and does not state dataset sizes, agent/model configurations, baselines beyond “corresponding base harnesses,” statistical uncertainty, cost or compute, or explicit limitations.
AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces
Research extraction

The supplied abstract supports the paper’s central claim that, for long-horizon tasks evaluated across models with comparable frontier capability, harness configuration can be a stronger determinant of performance than model choice.

  • Controlled-study design: The paper says it includes a “controlled variance decomposition,” but the supplied excerpt does not specify the tasks, models, harness conditions, controls, repetitions, sample sizes, or statistical method. The study design therefore cannot be verified in detail from this bundle.
  • Harness versus model effects: The abstract reports qualitatively that harness-induced variance can “substantially exceed” model-induced variance, but gives no numerical effect size, variance share, or uncertainty interval.
  • Ranking reversals: The paper explicitly claims that the evidence includes cases of model ranking reversal, but the excerpt does not identify the models, benchmarks, reversal direction, frequency, or magnitude.
  • Disclosure proposal: It proposes a harness-aware evaluation framework containing a disclosure standard and a variance-decomposition protocol, and warns that leaderboard comparisons without disclosed harness specifications are incomplete and potentially misleading. The excerpt does not enumerate the required disclosure fields.
Stop Comparing LLM Agents Without Disclosing the Harness
Research extraction

Verification: The post reports measured Jalapeño results on SemiAnalysis’s public InferenceX benchmark, comparing it with leading commercially available AI systems across high-throughput through highly interactive operating points. Across the three public models tested, OpenAI reports 1.5–1.9× more AI work per watt at peak throughput, 1.7–3.6× lower end-to-end latency, and 2.1–4.1× higher performance on highly interactive workloads.

Benchmark setup and power accounting

  • The evaluation targets a matched user experience: useful AI work per unit of power while satisfying the latency required by customers and interactive agents. InferenceX measures the full process of serving an AI request.
  • Results were normalized using each accelerator’s published chip-power rating. Jalapeño is rated at 700 W, but OpenAI says measured sustained power was at or below 550 W on the tested workloads.
  • The appendix specifies nominal 8k/1k input/output and STP. For GPT-OSS 120B, the package-TDP comparison is Jalapeño 700 W versus GB200 1,200 W; the DeepSeek R1 MXFP4 and Kimi K2.5 MXFP4 comparisons use GB300 at 1,400 W.

Reported model-level results

  • GPT-OSS 120B: The appendix reports approximately 1.9× higher peak mixed TPS/kW—85,448 versus 44,960—1.7× lower end-to-end latency—1.03 versus 1.80 seconds—and 2.7× lower minimum TBT—0.69 versus 1.87 ms, corresponding to 1,459 versus 535 tokens/user/s. It also reports approximately 53.7× more throughput per kW at the comparison system’s prior TBT: 22,935 versus 427 mixed/kW at 535.28 tokens/user/s.
  • DeepSeek R1 670B: The appendix reports approximately 1.7× higher peak mixed TPS/kW—19,641 versus 11,781—3.6× lower end-to-end latency—1.65 versus 5.99 seconds—and 4.1× lower minimum TBT—1.43 versus 5.90 ms, or 700 versus 169 tokens/user/s. It also reports approximately 104.3× more throughput per kW at the prior TBT: 12,258 versus 118 mixed/kW at 169.41 tokens/user/s.
  • Kimi K2.5 1T: The appendix reports approximately 1.5× higher peak mixed TPS/kW—18,195 versus 11,862—3.4× lower end-to-end latency—1.56 versus 5.31 seconds—and 3.8× lower minimum TBT—1.44 versus 5.48 ms, or 694 versus 182 tokens/user/s. It also reports approximately 56.1× more throughput per kW at the prior TBT: 6,744 versus 120 mixed/kW at 182.46 tokens/user/s.

Workload and model caveats

  • The cross-model headline is based on three public models—GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T—and the post says Jalapeño’s architecture works across models developed inside and outside OpenAI. OpenAI separately describes a wider advantage on frontier OpenAI models in internal testing, but characterizes that result only as a suggestion that the architecture becomes more valuable as workloads grow.
  • The benchmark spans both high-throughput serving and highly interactive, low-latency use, and the appendix reports peak and matched-throughput operating points rather than a single workload point.
  • Supporting each new model family still requires new kernels and model-specific optimization. The AI-programming claim concerns three open-weight models that were not in Jalapeño’s original production plan and were brought to high performance within two months; the reported 1.5–1.8× speedup applies only to selected GPT-OSS attention and mixture-of-experts blocks, explicitly not to the full model.

AI-assisted chip development

  • OpenAI says AI played a direct role in moving Jalapeño from initial design to tapeout in nine months by exploring implementations, shortening design, measurement, and verification loops, iterating on model workloads, and optimizing arithmetic circuits.
  • The chip was designed as a predictable programming target—local tensors, explicit communication, and predictable synchronization—so AI can optimize mapping, placement, scheduling, and coordination across the system.
  • Using Codex with GPT-Astra, the team brought the three out-of-plan open-weight models to high performance within two months; for selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5–1.8× faster than existing human-expert implementations.

Deployment timeline and status

  • OpenAI plans to begin deploying Jalapeño within its compute infrastructure by the end of the year. It calls this the first generation of a multigenerational roadmap, with Gen 2 deep in development and Gen 3 taking shape.
  • The stated pre-deployment work includes production qualification, software maturation, preparation to operate at scale, and validation across more models. OpenAI also says it will continue widely deploying NVIDIA and other partners’ accelerators for both training and inference.
Jalapeño’s first results show industry-leading speed and efficiency in AI inference | OpenAI
AI High Signal

A former employee of an unnamed outsource training provider alleged that RLVR environments for computer-use and MCP tasks were rushed and poorly representative of real software, while workers and models were encouraged to work around bugs to obtain verified rewards; human reviewers were discouraged from flagging broken environments because doing so reduced training-data volume. The source further alleged that this data was fed into models without a reliable signal distinguishing synthetic environments, and characterized the pressure to maximize volume over quality as an industry-wide practice.

Okay, so since I got laid off, I can actually explain a huge problem I saw from the inside with regard to industry practices on training …
AI High Signal
  • @gdb called AI for chip design “underrated.”
  • @beffjezos claimed that OpenAI is keeping its chip team small and using AI in an improvement loop, asking whether this could be an early stage of “hardware-in-the-loop RSI”; the claim is speculative and lacks concrete metrics.
ai for chip design is underrated [https://x.com/beffjezos/status/2092418951586390137](https://x.com/beffjezos/status/2092418951586390137) OpenAI decided to keep the chip team small and use AI for a an improvement loop Early stages of Hardware in the loop RSI? ![](https://pbs…
AI High Signal
  • LiquidAI, in partnership with Artificial Analysis, released Pipette, an open-source evaluation suite for on-device intelligence.
  • Pipette measures model quality, speed, latency, and memory use across phones, laptops, PCs, AI boxes, and embedded hardware, while evaluating the combined effects of the model, quantization, runtime, and device. Its benchmark warehouse includes more than 10,000 verified results across 35 model classes, seven quantization variants, llama.cpp runtimes, and four devices; the platform is designed to accept new devices, runtimes, model families, and quantization levels.
Today we release Pipette, a model evaluation suite for on-device intelligence, in partnership with [@ArtificialAnlys](https://x.com/Artif…
AI High Signal

Alibaba’s Qwen3.8-27B ranked #1 among open models and #7 overall in the Image-to-WebDev Arena. The Arena reports a score of 1,574, pricing of $0.40 per million input tokens and $3 per million output tokens, and performance on par with Kimi K3 (Max) at 2.8T parameters. Alibaba emphasized that the 27B model matches models roughly 100× larger.

Big news: Qwen3.8-27B by [@Alibaba_Qwen](https://x.com/Alibaba_Qwen) is now [#1](https://x.com/hashtag/1) among open models in the Image-… [#1](https://x.com/hashtag/1) among open models in the Image-to-WebDev Arena, [#7](https://x.com/hashtag/7) overall. 🚀 Thanks for the rec…
AI High Signal

Anecdotally, one student appears to be learning research practices from Codex and reportedly opened a meeting with a list of smoke tests he had run, suggesting AI coding agents may be influencing how researchers validate their work.

i have a student who appears to be learning how to do research from codex he started today's meeting with, and i am not joking, a list of…
AI High Signal

@jachiam0 argues that AI-risk attention is “overestimating AI-generated pathogens and underestimating designer drugs,” flagging future “ultrapotent, low-side-effect, low-cost” party drugs as an underappreciated risk if people gain the ability to design them.

We're overestimating AI-generated pathogens and underestimating designer drugs. As soon as people can start designing ultrapotent low-sid…
AI High Signal
  • Google’s TPU ecosystem still faces longstanding XLA compiler usability friction: nearly a decade in, Google is still pitching a smoother “eager mode.” The commentary notes this is especially notable because Google engineers were already using compilation through TensorFlow, underscoring how unintuitive XLA was.
  • A Google Hot Chips presentation was praised for openly explaining TPU architecture decisions and the reasoning behind them, including both strengths and weaknesses.
It’s crazy that TPUs are still overcoming the friction of the initial XLA compiler and pitching a smooth “eager mode” almost a decade in.… Google Hot Chips presentation is kick ass. So many interesting nuggets of information, the good and the bad. Why they made certain archit…
AI High Signal

A user reports that NousResearch’s Hermes agent installation instructions are incorrect when Computer Use (CUA) is disabled, and shares a PowerShell one-line workaround using the project’s install.ps1 script with -SkipComputerUse.

If you want to install [@NousResearch](https://x.com/NousResearch) hermes agent without CUA the instructions aren't correct on their site…
AI High Signal

AI can compress chip-design cycles, but it cannot solve semiconductor supply constraints.

hot chips hot take: you can compress design cycles with ai but you can’t fix supply
AI High Signal
  • T3 Code shipped a release with 380+ changes, including in-app PR reviews, npx t3 triage for debugging and fixing issues, and npx t3 connect support on Mac for turning a Mac into a remote development machine.
  • The update claims a 93.8%+ improvement in rendering long threads, an 80%+ reduction in thread data transferred and stored, and remote-server upgrades that are 5× faster and more reliable. It also improves Codex skill discovery, adds in-app visibility for OpenCode skills, enables Claude to discover project-level skills, and surfaces Claude Code compaction recommendations plus Codex permission requests for MCP/Computer Use.
Huge T3 Code release just dropped. Over 380 changes. Quick highlight reel of my favorites: - in-app PR reviews - \`npx t3 triage\` to deb…
AI High Signal

@saranormous satirizes 2026 startup branding, cycling through labels such as “neolab,” “product co,” “RLaaS,” “deployco,” and “humandata co” before repeatedly reframing them as “a neocloud”; it raises a supply-side question about where these new cloud ventures will obtain wafers and powered land, highlighting potential semiconductor and data-center infrastructure constraints.

2026 startup evolutions: we’re… a neolab! JK we’re a neocloud a product co! JK we’re a neocloud RLaaS! JK we’re a neocloud deployco! JK w…
AI High Signal
  • Richard Ngo argues that an AGI-safety “one ring” memeplex focused on controlling recursive self-improvement has narrowed strategic thinking among parts of Anthropic, OpenAI, and DeepMind; he says its urgency is often tied to vibes-based AGI timelines around 2027–2028 rather than concrete predictions.
  • He argues that AI has so far increased peak human agency and that research aimed at scaling to more capable systems can generally be conducted with publicly available models, challenging the assumption that researchers must remain inside AGI companies to do valuable work.
The AI safety community constructed a memeplex in which “taking AGI seriously” was a prerequisite for being a serious and good person. Wh… You don’t need to stay to do good research: work that aims to scale to much more intelligent systems can almost always be done on publicl…
AI High Signal

Spellbook Legal is hiring a VP of Customer Success to support its rapidly growing base of 5,000+ legal teams using AI to scale contract workflows.

We are hiring a VP Customer Success at [@SpellbookLegal](https://x.com/SpellbookLegal) to help our rapidly growing customer base of 5,000…
AI High Signal
  • Ollama v0.33 adds one-toggle configuration for Claude Desktop to use Ollama as a third-party gateway, supporting both cloud and local models.
  • Existing features remain available, while Ollama’s built-in web search and open models can be used for research tasks.
Ollama v0.33 is here! You can now easily configure Claude Desktop to seamlessly work with Ollama as a third-party gateway provider. One t… All the features that you are used to continue to work! Ollama’s web search comes baked in. Use open models to run research tasks. [https…
AI High Signal

In an AI-risk debate, @teortaxesTex argues that AGI/ASI being smarter than humans—including himself and Yudkowsky—would not imply convergence with either person’s values or decision theory.

This has direct relevance to my disagreement with Yud re: AI risk AGI/ASI will be smarter than me, and smarter than Yud this doesn't impl…
AI High Signal

A social-media post alleges that an Nvidia Jetson Orin chip was found in a Russian S-71M air-launched cruise missile; the post includes images but provides no further sourcing or technical detail.

Breaking: Nvidia jetson orin chip found in russian S-71M air-launched cruise missile Nvidia chips continue to be "not a car" ![](https://…