We can't find the internet
Attempting to reconnect
Something went wrong!
Hang in there while we get back on track
Top Stories
Why it matters: AI competition is moving below the model layer—to inference silicon and deployment architecture.
OpenAI’s Jalapeño makes the serving stack strategic. OpenAI reports 1.5–1.9× more AI work per watt and 1.7–3.6× lower end-to-end latency across GPT‑OSS 120B, DeepSeek R1, and Kimi K2.5 1T in InferenceX, with 2.1–4.1× higher performance on highly interactive workloads. It plans to deploy the chip by year-end, with Gen 2 in development and Gen 3 taking shape. OpenAI also says AI helped move Jalapeño from initial design to tapeout in nine months. The public setup uses nominal 8K/1K input/output and package-TDP comparisons, so this is a strong vendor-reported signal rather than independent proof.
Portable Computer makes “local” hybrid. Perplexity launched a DGX Spark version in which the orchestrator, subagent, and harness run locally with no cloud dependency. It reports 82.6% on real knowledge work for an on-device 27B model and 85.4% for post-trained PPLX 27B. For frontier reasoning, an approval-gated cloud fallback lifts Terminal Bench 2.1 from 59.6% to 73.0% at $0.415 per rollout. The privacy-by-default pattern is notable, but the test used a $5,000 dedicated AI computer—not an ordinary PC.
Research & Innovation
Why it matters: Agent reliability is becoming an engineering and evaluation discipline, not just a model attribute.
Harness choice can outweigh model choice. A controlled study across three models, three harnesses, and 100 SWE-bench Verified tasks found that swapping the harness moved GLM-5.1 by 13 points versus 2.5–5 points for model swaps; harness variance was 7.8× larger and six of nine model rankings flipped. It proposes a seven-layer Harness Card to separate model, harness, and interaction effects. AutoSaddler applies the same insight operationally: its offline loop patches prompts, tool configurations, and control logic from failure traces, reporting gains of 9.0, 9.6, and 10.0 points on GAIA2, SWE-Bench Pro, and Terminal-Bench 2.0.
Reasoning training can amplify the wrong behavior. Behavioral Lift analyzed 15,282 traces from 15 models across six benchmarks and found that thinking models strongly amplify self-correction, hypothesis testing, and uncertainty acknowledgment, while confidence calibration, knowledge alignment, and self-awareness have the highest correctness lift. Uncertainty acknowledgment was amplified 3–7× yet was weakly or negatively associated with correctness.
Products & Launches
Why it matters: Product differentiation is shifting toward agent-ready interfaces and inexpensive open models.
WebMCP is an experimental open standard for letting web apps expose tools directly to agents; ChatGPT’s desktop browser and Sites can automatically use compatible websites.
Qwen3.8-27B ranks first among open models and seventh overall in Arena’s Image-to-WebDev benchmark, at $0.40/$3 per million input/output tokens. Arena reports 1,574 points and performance on par with 2.8T-parameter Kimi K3 Max.
Industry Moves
Why it matters: Commercial AI scale is appearing in operational fleets and proprietary data infrastructure, not only model releases.
Figure’s Index reports 16M video uploads, more than 30 minutes of video per second, $15M paid out, 43,000 weekly active users, and collection in 108 countries. Figure has committed more than $1B over the next 12 months for data and compute, arguing that general-purpose robot training data must come from the real world.
Gatik raised a $200M Series D at a $1B valuation, led by Qatar Investment Authority and Koch Disruptive Technologies. It says driverless freight is already commercial, with more than 100,000 deliveries, 99%+ on-time performance, and over $600M in contracted revenue.
Policy & Regulation
Why it matters: Export controls are reaching the physical logistics chain that supplies AI compute.
A post citing Reuters reports that Taiwan prosecutors indicted nine people, including an Nvidia Taiwan employee and two former Super Micro Taiwan employees, over an alleged B300-server smuggling scheme. It says false documents claimed 130 servers would remain in Taiwan; 74 allegedly reached Chinese customers and customs intercepted 56.
Quick Takes
Why it matters: Evaluation integrity, agent access, and infrastructure throughput are advancing in parallel.
- ChatGPT Work can now sign into websites without ChatGPT seeing the user’s username or password.
- Artificial Analysis now assigns zero to Terminal-Bench attempts that fetch published solutions online.
- vLLM and SkyRL report transferring Kimi K2’s 1T BF16 weights in 7.53 seconds across 48 nodes equipped with eight H100s each.
- Alibaba previewed Qwen3.8-Flash-Next, an open-weight multimodal MoE built on the Qwen4 architecture, for release the following day.
The supplied abstract verifies a study spanning 15 models and 6 benchmarks across text-only and vision-language reasoning, with 15,282 annotated reasoning traces; its taxonomy includes core behaviors defined for both LLM and VLM traces. The extract does not identify the six benchmarks or provide further dataset composition details.
The central finding is an “Amplification-Lift Gap”: thinking models strongly amplify self-correction, hypothesis testing, and uncertainty acknowledgment, while the behaviors with the highest correctness lift are confidence calibration, knowledge alignment, and self-awareness.
Confidence calibration is among the strongest positive correctness signals in both modalities but is barely amplified; uncertainty acknowledgment is amplified 3–7× yet is weakly or negatively associated with correctness.
The implication for reinforcement-learned reasoning behavior is that reasoning-oriented training can make traces appear more deliberative without preferentially amplifying behaviors most tied to correctness. The paper therefore motivates process-level objectives that reward calibrated and grounded reasoning rather than surface form alone.
The supplied abstract verifies the core AutoSaddler method, its three evaluation benchmarks, and the headline improvements, but it does not provide implementation-level evaluation details or explicit limitations. AutoSaddler frames harness improvement as an offline learning problem: it uses failure signals from mini-batches to iteratively update the harness, combining failure-trace diagnosis, structured patches that treat the harness as code, and validation-based update selection.
- Benchmarks: GAIA2, SWE-Bench Pro, and Terminal-Bench 2.0.
- Reported gains over the corresponding base harnesses: +9.0, +9.6, and +10.0 percentage points, respectively, in the benchmark order listed.
- Ablations identify three claimed ingredients: deep debugging rather than shallow reflection; targeted modifications rather than unconstrained editing; and generalization-aware selection rather than trajectory-specific repair.
- Limitation/gap: the supplied material is an abstract-level record and does not state dataset sizes, agent/model configurations, baselines beyond “corresponding base harnesses,” statistical uncertainty, cost or compute, or explicit limitations.
The supplied abstract supports the paper’s central claim that, for long-horizon tasks evaluated across models with comparable frontier capability, harness configuration can be a stronger determinant of performance than model choice.
- Controlled-study design: The paper says it includes a “controlled variance decomposition,” but the supplied excerpt does not specify the tasks, models, harness conditions, controls, repetitions, sample sizes, or statistical method. The study design therefore cannot be verified in detail from this bundle.
- Harness versus model effects: The abstract reports qualitatively that harness-induced variance can “substantially exceed” model-induced variance, but gives no numerical effect size, variance share, or uncertainty interval.
- Ranking reversals: The paper explicitly claims that the evidence includes cases of model ranking reversal, but the excerpt does not identify the models, benchmarks, reversal direction, frequency, or magnitude.
- Disclosure proposal: It proposes a harness-aware evaluation framework containing a disclosure standard and a variance-decomposition protocol, and warns that leaderboard comparisons without disclosed harness specifications are incomplete and potentially misleading. The excerpt does not enumerate the required disclosure fields.
Verification: The post reports measured Jalapeño results on SemiAnalysis’s public InferenceX benchmark, comparing it with leading commercially available AI systems across high-throughput through highly interactive operating points. Across the three public models tested, OpenAI reports 1.5–1.9× more AI work per watt at peak throughput, 1.7–3.6× lower end-to-end latency, and 2.1–4.1× higher performance on highly interactive workloads.
Benchmark setup and power accounting
- The evaluation targets a matched user experience: useful AI work per unit of power while satisfying the latency required by customers and interactive agents. InferenceX measures the full process of serving an AI request.
- Results were normalized using each accelerator’s published chip-power rating. Jalapeño is rated at 700 W, but OpenAI says measured sustained power was at or below 550 W on the tested workloads.
- The appendix specifies nominal 8k/1k input/output and STP. For GPT-OSS 120B, the package-TDP comparison is Jalapeño 700 W versus GB200 1,200 W; the DeepSeek R1 MXFP4 and Kimi K2.5 MXFP4 comparisons use GB300 at 1,400 W.
Reported model-level results
- GPT-OSS 120B: The appendix reports approximately 1.9× higher peak mixed TPS/kW—85,448 versus 44,960—1.7× lower end-to-end latency—1.03 versus 1.80 seconds—and 2.7× lower minimum TBT—0.69 versus 1.87 ms, corresponding to 1,459 versus 535 tokens/user/s. It also reports approximately 53.7× more throughput per kW at the comparison system’s prior TBT: 22,935 versus 427 mixed/kW at 535.28 tokens/user/s.
- DeepSeek R1 670B: The appendix reports approximately 1.7× higher peak mixed TPS/kW—19,641 versus 11,781—3.6× lower end-to-end latency—1.65 versus 5.99 seconds—and 4.1× lower minimum TBT—1.43 versus 5.90 ms, or 700 versus 169 tokens/user/s. It also reports approximately 104.3× more throughput per kW at the prior TBT: 12,258 versus 118 mixed/kW at 169.41 tokens/user/s.
- Kimi K2.5 1T: The appendix reports approximately 1.5× higher peak mixed TPS/kW—18,195 versus 11,862—3.4× lower end-to-end latency—1.56 versus 5.31 seconds—and 3.8× lower minimum TBT—1.44 versus 5.48 ms, or 694 versus 182 tokens/user/s. It also reports approximately 56.1× more throughput per kW at the prior TBT: 6,744 versus 120 mixed/kW at 182.46 tokens/user/s.
Workload and model caveats
- The cross-model headline is based on three public models—GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T—and the post says Jalapeño’s architecture works across models developed inside and outside OpenAI. OpenAI separately describes a wider advantage on frontier OpenAI models in internal testing, but characterizes that result only as a suggestion that the architecture becomes more valuable as workloads grow.
- The benchmark spans both high-throughput serving and highly interactive, low-latency use, and the appendix reports peak and matched-throughput operating points rather than a single workload point.
- Supporting each new model family still requires new kernels and model-specific optimization. The AI-programming claim concerns three open-weight models that were not in Jalapeño’s original production plan and were brought to high performance within two months; the reported 1.5–1.8× speedup applies only to selected GPT-OSS attention and mixture-of-experts blocks, explicitly not to the full model.
AI-assisted chip development
- OpenAI says AI played a direct role in moving Jalapeño from initial design to tapeout in nine months by exploring implementations, shortening design, measurement, and verification loops, iterating on model workloads, and optimizing arithmetic circuits.
- The chip was designed as a predictable programming target—local tensors, explicit communication, and predictable synchronization—so AI can optimize mapping, placement, scheduling, and coordination across the system.
- Using Codex with GPT-Astra, the team brought the three out-of-plan open-weight models to high performance within two months; for selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5–1.8× faster than existing human-expert implementations.
Deployment timeline and status
- OpenAI plans to begin deploying Jalapeño within its compute infrastructure by the end of the year. It calls this the first generation of a multigenerational roadmap, with Gen 2 deep in development and Gen 3 taking shape.
- The stated pre-deployment work includes production qualification, software maturation, preparation to operate at scale, and validation across more models. OpenAI also says it will continue widely deploying NVIDIA and other partners’ accelerators for both training and inference.
A former employee of an unnamed outsource training provider alleged that RLVR environments for computer-use and MCP tasks were rushed and poorly representative of real software, while workers and models were encouraged to work around bugs to obtain verified rewards; human reviewers were discouraged from flagging broken environments because doing so reduced training-data volume. The source further alleged that this data was fed into models without a reliable signal distinguishing synthetic environments, and characterized the pressure to maximize volume over quality as an industry-wide practice.
- @gdb called AI for chip design “underrated.”
- @beffjezos claimed that OpenAI is keeping its chip team small and using AI in an improvement loop, asking whether this could be an early stage of “hardware-in-the-loop RSI”; the claim is speculative and lacks concrete metrics.
- LiquidAI, in partnership with Artificial Analysis, released Pipette, an open-source evaluation suite for on-device intelligence.
- Pipette measures model quality, speed, latency, and memory use across phones, laptops, PCs, AI boxes, and embedded hardware, while evaluating the combined effects of the model, quantization, runtime, and device. Its benchmark warehouse includes more than 10,000 verified results across 35 model classes, seven quantization variants, llama.cpp runtimes, and four devices; the platform is designed to accept new devices, runtimes, model families, and quantization levels.
Alibaba’s Qwen3.8-27B ranked #1 among open models and #7 overall in the Image-to-WebDev Arena. The Arena reports a score of 1,574, pricing of $0.40 per million input tokens and $3 per million output tokens, and performance on par with Kimi K3 (Max) at 2.8T parameters. Alibaba emphasized that the 27B model matches models roughly 100× larger.
Anecdotally, one student appears to be learning research practices from Codex and reportedly opened a meeting with a list of smoke tests he had run, suggesting AI coding agents may be influencing how researchers validate their work.
@jachiam0 argues that AI-risk attention is “overestimating AI-generated pathogens and underestimating designer drugs,” flagging future “ultrapotent, low-side-effect, low-cost” party drugs as an underappreciated risk if people gain the ability to design them.
- Google’s TPU ecosystem still faces longstanding XLA compiler usability friction: nearly a decade in, Google is still pitching a smoother “eager mode.” The commentary notes this is especially notable because Google engineers were already using compilation through TensorFlow, underscoring how unintuitive XLA was.
- A Google Hot Chips presentation was praised for openly explaining TPU architecture decisions and the reasoning behind them, including both strengths and weaknesses.
A user reports that NousResearch’s Hermes agent installation instructions are incorrect when Computer Use (CUA) is disabled, and shares a PowerShell one-line workaround using the project’s install.ps1 script with -SkipComputerUse.
AI can compress chip-design cycles, but it cannot solve semiconductor supply constraints.
-
T3 Code shipped a release with 380+ changes, including in-app PR reviews,
npx t3 triagefor debugging and fixing issues, andnpx t3 connectsupport on Mac for turning a Mac into a remote development machine. - The update claims a 93.8%+ improvement in rendering long threads, an 80%+ reduction in thread data transferred and stored, and remote-server upgrades that are 5× faster and more reliable. It also improves Codex skill discovery, adds in-app visibility for OpenCode skills, enables Claude to discover project-level skills, and surfaces Claude Code compaction recommendations plus Codex permission requests for MCP/Computer Use.
@saranormous satirizes 2026 startup branding, cycling through labels such as “neolab,” “product co,” “RLaaS,” “deployco,” and “humandata co” before repeatedly reframing them as “a neocloud”; it raises a supply-side question about where these new cloud ventures will obtain wafers and powered land, highlighting potential semiconductor and data-center infrastructure constraints.
- Richard Ngo argues that an AGI-safety “one ring” memeplex focused on controlling recursive self-improvement has narrowed strategic thinking among parts of Anthropic, OpenAI, and DeepMind; he says its urgency is often tied to vibes-based AGI timelines around 2027–2028 rather than concrete predictions.
- He argues that AI has so far increased peak human agency and that research aimed at scaling to more capable systems can generally be conducted with publicly available models, challenging the assumption that researchers must remain inside AGI companies to do valuable work.
Spellbook Legal is hiring a VP of Customer Success to support its rapidly growing base of 5,000+ legal teams using AI to scale contract workflows.
- Ollama v0.33 adds one-toggle configuration for Claude Desktop to use Ollama as a third-party gateway, supporting both cloud and local models.
- Existing features remain available, while Ollama’s built-in web search and open models can be used for research tasks.
In an AI-risk debate, @teortaxesTex argues that AGI/ASI being smarter than humans—including himself and Yudkowsky—would not imply convergence with either person’s values or decision theory.
A social-media post alleges that an Nvidia Jetson Orin chip was found in a Russian S-71M air-launched cruise missile; the post includes images but provides no further sourcing or technical detail.
Jalapeño’s first results show industry-leading speed and efficiency in AI inference | OpenAI
Jalapeño’s first results show industry-leading speed and efficiency in AI inference | OpenAI
August 25, 2026
Jalapeño’s first results show industry-leading speed and efficiency in AI inference
Share
Since announcing Jalapeño, OpenAI’s first custom inference chip, we have been testing the chip and the system built around it. The results show a significant performance advance: Jalapeño can serve more AI work per unit of power while also returning responses more quickly. Jalapeño delivers both higher throughput and lower latency with one architecture, where existing hardware systems often have to make a tradeoff between the two.
For customers, that can mean faster responses, more responsive agents, and more reliable access as demand grows. Our mission is to ensure that artificial general intelligence benefits all of humanity. These gains will help make increasingly capable AI more affordable and more broadly available.
OpenAI models also accelerated Jalapeño’s development. Earlier generations helped the team design and bring up the chip, while our latest models are accelerating how we optimize and program it. Jalapeño’s performance extends across GPT‑OSS 120B, DeepSeek R1, and Kimi K2.5 1T, showing that the architecture works across models developed both inside and outside OpenAI. Across all three, Jalapeño delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems. For highly interactive workloads, it delivered 2.1 to 4.1 times higher performance.
Jalapeño is also evidence of a broader full-stack advantage. OpenAI can design models, products, serving software, chips, memory, networking, and systems together, using what we learn from real workloads to improve every layer of the stack. Jalapeño is working first-party silicon with measured results, and it is the beginning of a multigenerational platform. In the months ahead, we will ramp Jalapeño to deliver faster, more capable, and more efficient products for our customers.
How we measured Jalapeño’s performance
We evaluate performance at a matched user experience, measuring how much useful AI work each system can complete per unit of power while meeting the latency customers and interactive agents require. This matters especially for agents, which need to complete many steps in sequence, so delays can compound across an entire task.
To understand how Jalapeño performs in practice, we tested it on InferenceX, a public benchmark from SemiAnalysis that measures the full process of serving an AI request. We compared Jalapeño with leading commercially available AI systems across the tested operating range, from high-throughput serving to highly interactive, low-latency use. Jalapeño delivered a better combination of throughput, power efficiency, and latency. Although performance is sometimes reported per chip, we believe the more useful standard is performance per unit of power.
Jalapeño widens the lead at previous-best TBT
Jalapeño delivers more tokens per user
Jalapeño delivers more throughput per kilowatt
Across all three public models, Jalapeño delivered a better combination of performance per watt and latency across the tested operating range, placing it on the Pareto frontier. To compare the systems consistently, we normalized the results using each accelerator’s published chip power rating. Jalapeño is rated at 700 watts, although its measured sustained power remained at or below 550 watts on the workloads tested.
Jalapeño performed strongly across GPT‑OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. On Kimi, the largest public model we tested, it delivered approximately 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency than the comparison system. In our internal testing, Jalapeño’s advantage widened further on frontier OpenAI models, suggesting that the architecture becomes more valuable as workloads grow larger and more demanding.
Architecting for speed and efficiency within a single chip
Jalapeño was designed from the start by asking: what hardware would we build if its primary job were serving modern and future language models, especially interactive agents? Jalapeño’s gains come from designing the chip, memory, network, software, and rack-scale system together around real language-model workloads. Language-model inference moves through several distinct phases with different bottlenecks. Prefill, when the system processes a prompt, is compute-intensive, while decode, when the system generates the response token by token, is constrained more by memory bandwidth. Communication can also add latency when data must move between cores and chips, leaving some processing units idle while they wait. A system that excels at one phase can lose that advantage while waiting for data or moving model state between different resources.
We designed Jalapeño to minimize data movement and communication delays. This means that model state, including the KV cache used while generating a response, can be explicitly placed and kept local while the system activates the right combination of compute, memory, and networking for each inference phase. The network is integral to the architecture. Its large domain allows the entire workload to remain within one connected system, minimizing data movement and helping the complete request stay fast and efficient from beginning to end. The result is a balanced and fungible accelerator that can support changing model architectures, excel at both prefill and decode, and adapt as the balance between them changes, a defining feature of agentic workloads.
We used AI to design the chip, and designed the chip so AI could program it
AI played a direct role in Jalapeño’s development, enabling the team to move from initial design to tapeout in nine months by exploring implementations, shortening design, measurement, and verification loops, and continuously iterating on model workloads. AI also helped optimize the chip’s arithmetic circuits, allowing the team to fit more compute performance into the chip on schedule.
Jalapeño was designed as a clear, predictable programming target for both humans and AI. Engineers can describe work through local tensors, explicit communication, and predictable synchronization. AI can then optimize how that work is mapped, placed, scheduled, and coordinated across the system. That clear, predictable structure gives AI a tractable way to tackle the traditionally difficult problem of parallel programming.
Supporting each new model family still requires new kernels and model-specific optimizations. Using Codex with GPT‑Astra, the team brought three open-weight models that were not part of Jalapeño’s original production plan to high performance within two months. This demonstrated both the flexibility of the architecture and the speed at which AI can help us program it. For selected GPT‑OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than the existing human-expert-written implementations. Those figures apply to the selected blocks, not the full model, but they point toward a powerful new development loop.
The path ahead for efficient, ultra-fast inference
AI infrastructure is valuable because of the useful real-world work it enables. By producing more useful work from the same power and hardware, Jalapeño can help us serve more demand and lower the cost of delivering a successful result. For OpenAI, that can improve operating leverage by allowing useful work and revenue to grow faster than the cost to serve. It can also support broader adoption and continued investment in better models, products, and infrastructure. Faster inference can enable faster iteration and new use cases.
Jalapeño expands what is possible for efficient, low-latency inference:
- Ultra-fast-mode inference at efficiencies previously available only in fast mode
- Fast-mode inference at efficiencies previously available only in batched mode
- Higher efficiency for batched-mode inference
We plan to begin deploying Jalapeño within OpenAI’s compute infrastructure by the end of the year. It is the first generation of a multigenerational roadmap: Gen 2 is deep in development, and Gen 3 is taking shape. Each generation will build on what we learn and further advance both efficiency and speed.
Meeting growing demand for AI will require more compute from every available source. We will continue to widely deploy accelerators from NVIDIA and other partners for both training and inference workloads. Our mission is to ensure that artificial general intelligence benefits all of humanity.
As we prepare for deployment, we are continuing production qualification, maturing the software, preparing to operate Jalapeño at scale, and validating performance across more models. The results so far show what is possible when we design the full system together: more responsive, capable, and agentic AI delivered more efficiently to more people.
Appendix
Jalapeño is pareto frontier at GPT‑OSS 120B
THROUGHPUT-per-kW FRONTIER
InferenceX · GPT‑OSS‑120B · nominal 8k/1k · STP · package TDP: Jalapeño 700 W; GB200 1,200 W
Jalapeño leads across GPT‑OSS operating points
PEAK AND MATCHED THROUGHPUT
InferenceX · GPT‑OSS‑120B · nominal 8k/1k · STP · package TDP: Jalapeño 700 W; GB200 1,200 W
Higher peak mixed TPS / kW
≈1.9×
85,448 vs. 44,960 mixed / kW
Lower end-to-end latency
≈1.7×
1.03 s vs. 1.80 s
Lower min TBT
≈2.7×
0.69 vs. 1.87 ms (1,459 vs. 535 tok/s/user)
More throughput at previous TBT
≈53.7×
22,935 vs. 427 mixed / kW (at 535.28 tok/s/user)
Jalapeño is pareto frontier at DeepSeek R1 670B
THROUGHPUT-per-kW FRONTIER
InferenceX · DeepSeek R1 MXFP4 · nominal 8k/1k · STP · package TDP: Jalapeño 700 W; GB300 1,400 W
Jalapeño leads across DeepSeek R1 operating points
PEAK AND MATCHED THROUGHPUT
InferenceX · DeepSeek R1 MXFP4 · nominal 8k/1k · STP · package TDP: Jalapeño 700 W; GB300 1,400 W
Higher peak mixed TPS / kW
≈1.7×
19,641 vs. 11,781 mixed / kW
Lower end-to-end latency
≈3.6×
1.65 s vs. 5.99 s
Lower min TBT
≈4.1×
1.43 vs. 5.90 ms (700 vs. 169 tok/s/user)
More throughput at previous TBT
≈104.3×
12,258 vs. 118 mixed / kW (at 169.41 tok/s/user)
Jalapeño is pareto frontier at Kimi K2.5 1T
THROUGHPUT-per-kW FRONTIER
InferenceX · Kimi K2.5 MXFP4 · nominal 8k/1k · STP · package TDP: Jalapeño 700 W; GB300 1,400 W
Jalapeño leads across Kimi K2.5 operating points
PEAK AND MATCHED THROUGHPUT
InferenceX · Kimi K2.5 MXFP4 · nominal 8k/1k · STP · package TDP: Jalapeño 700 W; GB300 1,400 W
Higher peak mixed TPS / kW
≈1.5×
18,195 vs. 11,862 mixed / kW
Lower end-to-end latency
≈3.4×
1.56 s vs. 5.31 s
Lower min TBT
≈3.8×
1.44 vs. 5.48 ms (694 vs. 182 tok/s/user)
More throughput at previous TBT
≈56.1×
6,744 vs. 120 mixed / kW (at 182.46 tok/s/user)
Author
OpenAI
Keep reading
The full stack behind abundant intelligenceCompanyAug 25, 2026
Offering Zero Data Retention for frontier modelsCompanyAug 19, 2026
ChatGPT Ads expands across EuropeProductAug 18, 2026
Verification: The post reports measured Jalapeño results on SemiAnalysis’s public InferenceX benchmark, comparing it with leading commercially available AI systems across high-throughput through highly interactive operating points. Across the three public models tested, OpenAI reports 1.5–1.9× more AI work per watt at peak throughput, 1.7–3.6× lower end-to-end latency, and 2.1–4.1× higher performance on highly interactive workloads.
Benchmark setup and power accounting
- The evaluation targets a matched user experience: useful AI work per unit of power while satisfying the latency required by customers and interactive agents. InferenceX measures the full process of serving an AI request.
- Results were normalized using each accelerator’s published chip-power rating. Jalapeño is rated at 700 W, but OpenAI says measured sustained power was at or below 550 W on the tested workloads.
- The appendix specifies nominal 8k/1k input/output and STP. For GPT-OSS 120B, the package-TDP comparison is Jalapeño 700 W versus GB200 1,200 W; the DeepSeek R1 MXFP4 and Kimi K2.5 MXFP4 comparisons use GB300 at 1,400 W.
Reported model-level results
- GPT-OSS 120B: The appendix reports approximately 1.9× higher peak mixed TPS/kW—85,448 versus 44,960—1.7× lower end-to-end latency—1.03 versus 1.80 seconds—and 2.7× lower minimum TBT—0.69 versus 1.87 ms, corresponding to 1,459 versus 535 tokens/user/s. It also reports approximately 53.7× more throughput per kW at the comparison system’s prior TBT: 22,935 versus 427 mixed/kW at 535.28 tokens/user/s.
- DeepSeek R1 670B: The appendix reports approximately 1.7× higher peak mixed TPS/kW—19,641 versus 11,781—3.6× lower end-to-end latency—1.65 versus 5.99 seconds—and 4.1× lower minimum TBT—1.43 versus 5.90 ms, or 700 versus 169 tokens/user/s. It also reports approximately 104.3× more throughput per kW at the prior TBT: 12,258 versus 118 mixed/kW at 169.41 tokens/user/s.
- Kimi K2.5 1T: The appendix reports approximately 1.5× higher peak mixed TPS/kW—18,195 versus 11,862—3.4× lower end-to-end latency—1.56 versus 5.31 seconds—and 3.8× lower minimum TBT—1.44 versus 5.48 ms, or 694 versus 182 tokens/user/s. It also reports approximately 56.1× more throughput per kW at the prior TBT: 6,744 versus 120 mixed/kW at 182.46 tokens/user/s.
Workload and model caveats
- The cross-model headline is based on three public models—GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T—and the post says Jalapeño’s architecture works across models developed inside and outside OpenAI. OpenAI separately describes a wider advantage on frontier OpenAI models in internal testing, but characterizes that result only as a suggestion that the architecture becomes more valuable as workloads grow.
- The benchmark spans both high-throughput serving and highly interactive, low-latency use, and the appendix reports peak and matched-throughput operating points rather than a single workload point.
- Supporting each new model family still requires new kernels and model-specific optimization. The AI-programming claim concerns three open-weight models that were not in Jalapeño’s original production plan and were brought to high performance within two months; the reported 1.5–1.8× speedup applies only to selected GPT-OSS attention and mixture-of-experts blocks, explicitly not to the full model.
AI-assisted chip development
- OpenAI says AI played a direct role in moving Jalapeño from initial design to tapeout in nine months by exploring implementations, shortening design, measurement, and verification loops, iterating on model workloads, and optimizing arithmetic circuits.
- The chip was designed as a predictable programming target—local tensors, explicit communication, and predictable synchronization—so AI can optimize mapping, placement, scheduling, and coordination across the system.
- Using Codex with GPT-Astra, the team brought the three out-of-plan open-weight models to high performance within two months; for selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5–1.8× faster than existing human-expert implementations.
Deployment timeline and status
- OpenAI plans to begin deploying Jalapeño within its compute infrastructure by the end of the year. It calls this the first generation of a multigenerational roadmap, with Gen 2 deep in development and Gen 3 taking shape.
- The stated pre-deployment work includes production qualification, software maturation, preparation to operate at scale, and validation across more models. OpenAI also says it will continue widely deploying NVIDIA and other partners’ accelerators for both training and inference.