ZeroNoise Logo zeronoise
Post
Astra and Fable Tie on a Tougher AI Index—Reliability Remains the Gap
4 min read
905 docs
Artificial Analysis’s new private-workflow evaluation puts Astra and Fable at the top, while end-to-end tests and the day’s infrastructure moves show that dependable execution, inference cost, and compute scale matter more than a single leaderboard score.

Top Stories

Why it matters: Frontier-model leadership is shifting from benchmark answers to costed, guardrailed work that can survive deployment.

Artificial Analysis reset its leaderboard around harder-to-game workflows. v4.3 upgrades Terminal-Bench to 4.0 and adds AutomationBench-AA, a private 657-workflow test across simulated Gmail, Slack, Salesforce and Jira; private-task/answer weight rises to 45%. GPT-6 Astra and Claude Fable 5.1 tie at 53, but Astra’s average cost is $3.26 per task versus Fable’s $7.63. On AutomationBench, Astra scores 68.5% yet completes every objective without a guardrail violation in only 41.6% of workflows. The second number is the better deployment signal: broad competence still does not equal reliable execution.

Real-world deployment remains much harder than leaderboard work. A separate end-to-end benchmark gives an agent business records, a client, a production API, an inherited codebase and hard cost/model limits, then scores its deployed agent against held-out users. Claude Opus 5 under Claude Code passed 23.9% of evaluations versus 82.2% for an expert human; failures included shallow record use, little client questioning and shipping the first runnable design. It is not an Astra test, but it makes index leadership a poor proxy for workplace readiness.

OpenAI’s capability push now carries an internal caution signal. A post quoting the company’s chief scientist says it is time for “extreme caution,” calls racing at all costs absurd, and urges governments to prioritize international coordination. That is a strategic counter-signal to the period’s acceleration.

Research & Innovation

Why it matters: AI research is turning its own software and data pipelines into objects that agents can regenerate, test and improve.

An AI-native performance-modeling paper makes design documents the source of truth. The Google DeepMind/MIT work described in the feed keeps almost no code on its main branch; coding sub-agents regenerate the implementation from a directed graph of natural-language documents. Worked examples plus a recursive operator IR and SymPy cost layer anchor the process, and the post reports round-off-precision reproduction of hand-audited models, including DeepSeek-V3 serving on a TPU slice.

Open models are improving at small scale, but unevenly. OpenBMB’s 2.6B-parameter MiniCPM5-2B, released under Apache 2.0, scores 15 on Artificial Analysis v4.2—the highest among open-weight models below 4B—and leads that size band on GDPval’s agentic Elo at 831. It uses 19k output tokens per task, but scores 9% on Humanity’s Last Exam, 9% on Terminal-Bench and 0% on CritPt.

Products & Launches

Why it matters: Product competition is now about latency and persistent context, not just model quality.

Sol-H3 crosses playback speed. NVIDIA’s Sol team and MiniMax released an open-source MiniMax-H3 inference stack that generates five seconds of 1344×768 video with stereo audio in 1.653 seconds on 8× B300 GPUs, with up to a 15.54× speedup versus Base H3. The full-profile comparison uses four rather than 49 DiT forwards; the code is Apache 2.0 and available through the Reactor API.

ChatGPT Work is productizing style memory. OpenAI says it can learn a user’s phrases, sign-off and capitalization from connected Gmail, Drive, Slack and SharePoint, then apply that style to future writing; the feature is available on paid plans with Work access.

Industry Moves

Why it matters: Physical AI is being scaled through dedicated data and compute pipelines, while China is trying to co-design chips, memory and manufacturing.

Figure is treating physical AI as a data-and-compute build-out. Its Index reports 16 million uploaded videos and 30 minutes of video per second, alongside a commitment to spend more than $1 billion on data and compute over 12 months. Its Nscale partnership covers up to 100,000 Vera Rubin GPUs, an initial $3.5 billion compute commitment intended to exceed $6 billion, and deployment from the second half of 2027 to train Helix.

China’s stack is moving toward workload-specific verticalization. A Bloomberg-linked analysis, explicitly conditional on the report’s accuracy, says DeepSeek is planning 160,000 Huawei chips and frames the order as co-optimization of memory and serving software for KV-cache-bound inference; production capacity may determine the timeline. Separately, a post citing the Financial Times says Huawei is coordinating domestic DUV suppliers, with 12 machines planned by year-end, while Zeiss optics and high-power light sources remain bottlenecks.

Quick Takes

Why it matters: Operating data and infrastructure prices are becoming as informative as model announcements.

  • WearableQA: Meta’s benchmark uses 4,084 questions from 200 users’ longitudinal wearable data; reported model accuracy spans 19.6%–72.9%, with cross-signal reasoning still difficult.
  • Agent adoption: Overall PyPI downloads fell 4.5% in August, but mcp-types rose 425% to 61.3 million downloads and OpenHands rose 128% to 2.52 million.
  • Compute demand: H100 rental prices rose 22% month over month to $3.28 per hour despite the chip being three years old.
  • Workflow economics: An Astra-orchestrated, no-human-label segmentation run took 1h 56m and 122.54 million tokens; its comparison used only 31 correlated validation frames and two diagnostic photos, so it remains a workflow demonstration rather than a general result.
Astra and Fable Tie on a Tougher AI Index—Reliability Remains the Gap
Research extraction
elvis

Significance: The post describes a potentially important shift in ML performance-modeling software: natural-language design documents, organized as a directed graph, become the source of truth, while coding agents regenerate the implementation instead of humans maintaining a persistent codebase. The motivation is that hardware and model generations repeatedly invalidate existing performance-modeling abstractions.

Why the approach may be robust: Worked examples in the design documents provide in-context demonstrations for the generating agents, while a minimal recursively defined operator IR and symbolic SymPy cost expressions provide a stable formal anchor.

Reported result: The post claims that regenerated implementations reproduce hand-audited reference models to round-off precision, including a DeepSeek-V3 serving model on a TPU pod slice. This is the strongest evidence offered for practical significance, but it is reported as a result rather than independently substantiated in the supplied material.

Caveats: The stated conceptual caveat is that the approach rests on the premise that performance-modeling abstractions are invalidated by each hardware and model generation; the post does not establish how universal that premise is. The supplied post gives no explicit benchmark scope, baselines, failure cases, regeneration costs, or limitations beyond the reported reproduction claim, so its evidence should not be generalized beyond the described examples without the paper itself.

// Design Docs Are All You Need // Banger paper from Google DeepMind, MIT, and colleagues. What a genuinely strange and interesting paper…
Research extraction
Zhihu Frontier

Assessment: usable only for a cautious, attributed industry-moves item—not as a confirmed order or independently verified technical rationale. The source presents a reported 160,000-chip DeepSeek order and explicitly conditions it on Bloomberg’s accuracy, framing it as a bet on coordinating domestic chips, memory, and serving software rather than simply replacing NVIDIA. It links to Bloomberg, but the supplied bundle contains only this X post and does not reproduce Bloomberg’s reporting or provide independent confirmation.

  • Order status: Describe it as “Bloomberg-reported” or “reported,” not finalized, delivered, or operational. The post itself cautions that production capacity may determine the timeline.
  • Memory-bandwidth rationale: The post gives a technically coherent claimed rationale: it lists 950PR at 96 GB of memory and 950DT at 144 GB, 4 TB/s memory bandwidth, and 2 TB/s interconnect bandwidth; it then argues that Decode repeatedly reads the growing KV cache, making memory bandwidth a bottleneck for large models and long contexts.
  • Evidence quality of that rationale: Treat the explanation as analysis attributed to Zhihu contributor “AI解码师,” not as a disclosed DeepSeek or Huawei procurement justification. The post further claims Huawei’s HiZQ 2.0 strategy could reduce dependence on conventional HBM and that inference is a lower-risk test, but supplies no benchmark, purchase document, company statement, or reproduced technical source in the bundle.
  • Recommended wording: “Bloomberg reportedly said DeepSeek plans to order 160,000 Huawei AI chips; an accompanying analysis argues the choice reflects memory-bandwidth and KV-cache constraints in inference.” Avoid presenting the chip specifications or the memory rationale as independently established facts unless separately corroborated.
DeepSeek’s Reported 160,000-Chip Order Is Really a Bet on Memory If Bloomberg’s report is accurate, DeepSeek is not simply replacing NVID…
Research extraction

Verification: Figure reports that Index had crossed 264,000 downloads across 108 countries, more than 44,000 weekly active users, and was processing 30 minutes of video uploads per second. Creators had uploaded over 16 million videos, the throughput was characterized as 4.9 years of human work uploaded per day, and Figure had paid Creators $15 million to date. The dataset’s reported breadth was 373 unique tasks, 1,146 unique manipulated objects, and 116 unique environments per 1,000 hours collected. Figure also stated that it was committed to spending more than $1 billion on data and compute over the following 12 months.

Nscale partnership terms: The announcement covers up to 100,000 NVIDIA Vera Rubin GPUs, with initial deployment targeted for the second half of 2027 in Barstow, Texas. It describes an initial compute commitment of $3.5 billion, with intent to scale beyond $6 billion. Nscale is also stated to be making a strategic investment in Figure, but the supplied announcement does not disclose that investment’s dollar amount; the $3.5 billion figure is explicitly identified as compute, not the strategic investment.

Strategic purpose: Figure says it is constrained by the data and compute needed to train Helix, and that the Nscale partnership supplies the compute runway to train the next generation of AI models for general robotics. The parties will also explore using humanoids to scale Nscale’s supply chain, while Figure describes the reliable-compute access as supporting its goal of bringing humanoid robots into every home.

Data-rate caveat: The Index announcement consistently states 30 minutes of video uploads per second, but the Nscale announcement describes Index as generating 35 minutes per second. The supplied sources do not reconcile this discrepancy.

Introducing Index: Building The World’s Largest and Most Diverse Physical Dataset Figure and Nscale Sign Strategic Partnership For Up to 100,000 GPUs on the NVIDIA Vera Rubin Platform
AI High Signal
  • GPT-6 Astra is praised for doing things the poster never thought a model could do, but criticized for also doing some of the stupidest things they have seen from a model; Fable 5.1 is described as generally doing what is asked.
  • The post conditionally forecasts GPT-6.1 Astra as potentially “the greatest model ever made” and “world-changing” if a follow-up improves GPT-6 as GPT-5.5’s follow-up did; it provides no benchmark or launch details.
GPT-6 Astra has done incredible things I never thought a model could do. It has also done some of the stupidest things I've ever seen a m… Alternative framing: GPT-6.1 Astra will be, by far, the greatest model ever made. If they can do to GPT-6 what they did to GPT-5.5 with t…
AI High Signal
  • Lumentum is reportedly using the MOPA approach for DWDM arrays in work with Nvidia, a development relevant to co-integrated optical interconnects. The cited ECTC 2026 paper covers the design and packaging of a DWDM CW-DFB laser array for co-integrated optical interconnects.
Well well, isn't this something! Lumentum is using the MOPA approach for DWDM arrays, with Nvidia of all people! My latest report talks a… [@vikramskr](https://x.com/vikramskr) lumentum is also doing MOPA with nvidia, this is from ECTC 2026 [Design and Packaging of a DWDM CW-…
AI High Signal

Theo describes GPT-6 Astra as unusually inconsistent: it has achieved things he “never thought a model could do” while also producing some of the “stupidest” behavior he has seen. He contrasts it with Fable 5.1, which he says generally follows instructions reliably. In a follow-up, he characterizes Astra as swinging between seemingly extraordinary performance and behavior resembling a distilled Gemini Flash model.

GPT-6 Astra has done incredible things I never thought a model could do. It has also done some of the stupidest things I've ever seen a m… I've never encountered a model quite as spiky as GPT-6 Astra. Sometimes it feels like they invented God, other times it feels like they d…
AI High Signal
  • Huawei is reportedly taking a deeper role in China’s domestic DUV lithography supply chain, investing in equipment and component companies and coordinating suppliers, fabs, and toolmakers; a Bernstein analyst described Huawei as the project manager of China’s DUV effort.
  • Shanghai 宇量昇’s domestically developed advanced DUV equipment is being tested at SMIC and Huawei production lines, with 12 machines planned for production by year-end. The initial focus is simpler chips, with eventual use targeted at advanced AI-chip manufacturing.
  • China’s DUV effort still depends on overseas inputs: Zeiss supplies the projection lenses and high-power light sources remain a major bottleneck. Huawei has invested in 至期光子 and 科益虹源 to localize those components.
华为正在更深地介入国产光刻机供应链。英国《金融时报》称,华为不仅投资相关设备和零部件公司,还在协调供应商、晶圆厂和整机厂商,推动国产 DUV(深紫外光刻机)进入实际产线。Bernstein 分析师直接把华为形容为中国 DUV 项目的「项目管理者」。 其中进展最快的是上海宇量…
AI High Signal
  • If Bloomberg’s report is accurate, DeepSeek is planning a reported 160,000-chip Huawei order—about 16× China’s first publicly reported 10,000-chip Ascend cluster—representing a workload-specific domestic alternative to NVIDIA rather than merely a fallback; production capacity could determine the deployment timeline.
  • The reported architecture pairs Ascend 950PR, with 96 GB of memory for compute-heavy prefill, with 950DT, offering 144 GB of HiZQ 2.0, 4 TB/s memory bandwidth, and 2 TB/s interconnect bandwidth for decode and training. The strategy targets LLM decode, where repeatedly reading the growing KV cache makes memory bandwidth a bottleneck; larger memory could keep model weights on fewer cards and reduce communication overhead.
  • The broader implication is a push to co-optimize domestic chips, memory, packaging, and serving software around inference workloads, potentially reducing dependence on conventional HBM suppliers; inference is framed as a lower-risk proving ground before large-scale training.
DeepSeek’s Reported 160,000-Chip Order Is Really a Bet on Memory If Bloomberg’s report is accurate, DeepSeek is not simply replacing NVID…
AI High Signal

A social-media post claims that Anthropic canceled a roughly $6 billion acquisition of Israeli AI startup Decart after months of due diligence, shortly before Decart’s IPO; it gives no reason for the cancellation. Decart is described as an approximately 100-person, NVIDIA-backed company whose inference engine delivers a claimed 8× speed advantage over industry alternatives; the post says NVIDIA offered about $8 billion, while Decart preferred Anthropic’s pre-IPO stock.

BREAKING: ANTHROPIC ABANDONS \~$6 BILLION ACQUISITION OF DECART AI >be decart >israeli AI startup \~100 people >build DOS inference engin…
AI High Signal
  • Jensen Huang claimed OpenAI’s GPT-6 Astra was trained on approximately 100K+ NVIDIA Grace Blackwell NVLink72 systems, announced that “AGI has arrived,” and said another 400K GPUs were coming online.
  • The post’s author disputes that AGI has arrived, arguing that true AGI requires mastery of the physical world, including robot-level capabilities and eventually self-improving hardware—not only software performance in virtual environments.
[@ChaseLochmiller](https://x.com/ChaseLochmiller) [@OpenAI](https://x.com/OpenAI) GPT-6 Astra, trained on \~100K+ NVIDIA Grace Blackwell … Some claim that AGI is here now. Ridiculous. No AGI without mastery of the real world! No true self-improvement without self-improving ha…
AI High Signal

Online evaluation remains a claimed bottleneck for AI applications: @Shahules786 argues that, if online evals had worked, RSI would already have appeared at the application layer before the model layer. A linked post by @boristane adds that agent observability has not yet been solved.

If online evals had ever worked, we’d have RSI at the application layer before the model layer. We don’t. [https://x.com/boristane/status… nobody has yet to figure out agents observability
AI High Signal
  • Paid AI consumption in households is growing steadily but has “barely scratched the surface”; Daniel Newman sees substantial revenue upside at the current 2.2% level.
Paid AI consumption is growing steadily in households but has barely scratched the surface. A lot of revenue upside from here at just 2.2…
AI High Signal

Goldman Sachs forecasts China’s AI-chip shipments will reach 5.9 million units in 2026, 21.2 million in 2028, and 39.2 million in 2030, implying rapid expansion of the country’s AI-hardware supply.

Goldman Sachs forecasts that China's AI chip shipments will reach 5.9 million / 21.2 million / 39.2 million units in 2026E / 2028E / 2030…
AI High Signal

CPO laser architectures: Lumentum uses a single high-power cavity producing about 400 mW or more, while Sumitomo uses a lower-power continuous-wave DFB laser followed by an amplifier to exceed 400 mW. The approaches involve different trade-offs, with the analysis highlighting considerations relevant to evaluating Lumentum’s position and investor watchpoints.

Part 3 of the series on UHP lasers for CPO is out. In this post, we talk about the Master Oscillator Power Amplifier (MOPA) approach. Lum…
AI High Signal
  • Anthropic’s transparency posture is contested. Shayne Redford argues the lab may be more transparent than OpenAI in key areas, citing its detailed Economic Index, Researcher Access Program for external real-user log analysis, Transparency Hub, and active Alignment Science and Frontier Red Teaming blogs. He says public data, reports, and access programs let outside observers independently assess AI safety impacts.
  • Jachiam’s counterargument is that Anthropic selectively engages publicly and keeps internal disagreements and shortcomings largely private, potentially fostering overconfidence in its safety posture. He argues a healthy safety culture must remain preoccupied with the possibility of organizational failure rather than assuming Anthropic is less vulnerable than rival labs.
This is a thoughtful and appreciated post, but (from my brief tenure) I question that Anthropic is less transparent than OpenAI in many o… Because Anthropic has a cultural belief that transparency leads to pressure that would decrease their ability to solve problems internall…
AI High Signal

A user demonstration showed Codex turning a desk photo into a miniature racing game: it recreated the desk in Blender, built a browser game with Three.js, and included a playable driving mode.

Uploaded a photo of my desk and asked Codex to build a miniature racing game from it. I'm calling it Desk Grand Prix 🏎️ Codex re-created t…
AI High Signal

H100 rental prices reportedly rose 22% month over month to $3.28 per hour, despite the chip being three years old—an indication that demand for AI training capacity may be slowing its economic depreciation. @iScienceLuvr presented the price increase as support for NVIDIA’s earlier decision to revise its GPU depreciation timeline, though no accounting details were provided.

The H100 is a three-year-old training chip. Its rental price is up 22 percent on the month, to $3.28 an hour. Every depreciation schedule… remember when last year, in one of the earnings reports, NVIDIA changed the depreciation timeline of GPUs and everybody got mad and compl…
AI High Signal

ChrisGPT says he is collaborating with Matt Shumer on an open-world New York City game and believes improving AI models are enabling increasingly ambitious game creation, with no apparent ceiling yet for scale and complexity. The post provides no performance metrics, release timing, or product details.

I’m happy to say I’ve been collabing with who I think is the game demo GOAT, [@mattshumer_](https://x.com/mattshumer_), on his open-world…
AI High Signal
  • An announcement claims IRON has launched the world’s first automated production line for advanced general-purpose humanoid robots, with robots autonomously mass-producing robots; it says the first robot completed assembly and walked off the line, and that the robots are ready to scale into dangerous, repetitive, or undesirable work.
  • The claim is tempered by skepticism about whether IRON’s software is advanced enough to contribute meaningfully to self-assembly.
Today, we launched the world’s first automated production line for advanced general‑purpose humanoid robots, using robots to mass‑produce… Is this anything? IRON's hardware seems decent but when has their software progressed to the point that it can meaningfully contribute to…
AI High Signal
  • Huawei’s updated Tau Law paper reports that Logic Folding reduced clock buffers in one processing module from roughly 43,600 to 19,000—a 56% reduction—by placing related circuit blocks on adjacent active layers and replacing long horizontal wires with short inter-layer links.
  • The saved timing slack can be traded for higher frequency at the same voltage or lower voltage at the same frequency. The approach requires joint optimization of 3D placement, routing, clock networks, power delivery, and thermal behavior; an analysis infers a working Logic Folding design and verification flow from reported silicon results, but its toolchain maturity, product scalability, and real-world gains remain unverified.
Huawei Updates Tau Law: Cutting Power Before It Becomes Heat In one processing module, Logic Folding cut clock buffers from roughly 43,60…