We can't find the internet
Attempting to reconnect
Something went wrong!
Hang in there while we get back on track
GDPx: a frontier benchmark where safety classifiers change the rankings
SemiAnalysis released GDPx, a benchmark of agentic capability across what it calls "the most valuable domains." It describes GDPx as frontier-level and says even the best model scores just 12% . The domains were chosen on purpose. SemiAnalysis expects the first trillion dollars of AI revenue to concentrate in sectors where frontier intelligence pays off most, so it included chip design, cyber and bio rather than government, real estate or insurance. It argues that benchmarks weighted by share of GDP or by occupational taxonomies are a mistake .
The main caveat: SemiAnalysis says Opus 5.5 and GPT-6 Astra have "sandbagged overall scores due to their safety classifiers." With cyber, biology and alignment excluded, the three domains where those classifiers can trigger, the ranking matched its own subjective impression of the models . It also names Muse Wonder Full as the best available cyber model "by a meaningful margin," and says classifiers matter more for model choice than people assume . The practical upshot is that one GDPx headline number can mislead buyers in regulated or dual-use domains.
Early reactions:
- Dylan Patel issued an "official apology to Google," saying Gemini 4 Carbon is spikier across domains but "really good in a few areas" .
- Elie Bakouch found Sol 6.1 and Astra scoring very similarly, Carbon beating Argon across the board, and Sonnet strong on SWE-style tasks . Once he removed the safeguard-affected benchmarks, Carbon looked very cheap for its performance and Sol 6.1 even cheaper .
- Openness. One critic, posting in a self-described joke, objects that GDPx is not open source, so its data and verifier code can't be audited .
New models from both labs appear close
Anthropic. Mythos 5.5 appears in the HTML source of Anthropic's official site, according to a tip passed to kimmonismus . A follow-up says the entry was added within the last few days . The tagline "our most capable model class for cybersecurity and biology research" refers to all Mythos models, not 5.5 specifically . There is no confirmed launch date.
OpenAI. Some Codex accounts now list gpt-6.2-astra as a separate line in usage analytics, while the model picker still shows GPT-6 Astra. The poster says multiple users see the same thing and calls it a gray rollout . Kimmonismus expects a release "very soon," which is speculation . Teortaxes argues that Astra 6.0 was already stronger than Opus 5.5 but lacked product-oriented post-training, and that 6.2 seems to fix this . One user's complaint shows the cost problem: on corporate plans they exhaust Astra's weekly limit in a day, so they now use it only to review Opus 5.5's work .
On capability tracking, Teortaxes reports that DeepSeek V4.1F-1017 scores 159 on Epoch's Capabilities Index (ECI). That ties GPT 5.5 and is 10 points above the original V4-Pro, with Astra another 10 points higher . He argues the index needs a refresh because Astra has saturated many of its evals . Scaling01 counters that Chinese models still haven't reached Mythos Preview, at about 161, after nearly seven months .
DeepSeek's Super-agent: limits stated, critics speak up
Last period's Super-agent release reached Bloomberg, which framed it as a challenge to Claude Code . Teortaxes noted the coverage came even though it was a "half-baked prototype" with no English announcement . Tianyi Cui, one of the article's final reviewers, set out its limits:
- "Super" refers to a layer above the working agent, not to greater intelligence .
- DeepSeek Harness had no long-term memory across sessions as of his post .
- The prototype implements only part of the design and has been validated only in a small, trusted internal environment .
Opinion split. Teortaxes calls the design "something of a hot mess" that leans on crutches to work around hallucinations . Mario Zechner (@badlogicgames) says Earendil is moving toward the same pattern: a single long-lived agent, recursive delegation, and event-driven triggers .
Cybersecurity: exploit data offers, an exploit anecdote, a fake leak
- Exploit-data offer. Jun Song says an anonymous black-hat group offered "an astronomical amount" for post-training open-weight models on exploit data, and claimed an attack corpus larger than anything he had seen. He says a white-hat GLM-5.3 project using under 1% of that volume "outperformed frontier models by a huge margin." He declined, and warns someone else will accept .
- CVE-2026-98432. One researcher calls this Linux TCP use-after-free severe: Go 1.24+ servers are exposed because MPTCP is on by default. He adds that it is likely exploitable mainly by an attacker on the same network . His Mythos run spent about $2k over three days without producing a reliable proof of concept, and GPT-5.6 Sol couldn't get a memory leak. Anthropic, by contrast, reportedly needed 40 minutes. He suspects Anthropic used an unreleased model or an expert exploit developer. This is one person's anecdote .
- Fake "Numbat" leak. A Hugging Face repo claiming to hold a 14T-parameter model in the Mythos class drew attention . Parts of it exactly match randomly generated numbers, which suggests the weights were never trained and the leak is likely fake .
Compute economics
- NVIDIA inference results. SemiAnalysis reports an NVIDIA AgentX submission reaching over 1,000 tok/s/user on 700B+ models, using software written by agent swarms. On Rubin NVL72 it claims 20.9× better performance per TCO than GB300 NVL72 at 325 tok/s/user . The gain relies on Mooncake Store, which pools KV cache in Vera LPDDR5X memory across 18 trays over NVLink instead of RDMA, for over 10× the bandwidth .
- China pricing. Citing industry contacts, one poster says Blackwell prices in China tripled in four months, with an eight-GPU server at RMB 16M . Teortaxes converts that to about $300K per B300 and estimates roughly a year to break even, even at DeepSeek's revenues .
- US buildout. SemiAnalysis says US datacenter capacity for 2027 is largely locked in, but a prolonged Texas permit freeze is now the biggest single risk for 2028 .
Briefly
- Karpathy's HackerTube. Claude filtered 43M Hacker News comments down to 6,454 repeatedly recommended videos, in about 2 hours with Opus 5.5 . His only manual steps were an API key, a VPS and buying the domain; Claude secured, deployed and optimized the site .
- Sakana Sawara. Sakana AI is training its own language-model series from pretraining. Its first model, Sawara Small, is targeted for spring 2027 . David Ha calls it an agent-native LLM built at Sakana's RSI Lab .
- Suleyman on consciousness. Mustafa Suleyman says he has seen no evidence that AIs are conscious, and that he would call for a halt if consciousness emerged during training . He argues AIs should be designed to act as non-conscious, and calls for safety information sharing among labs .
- MiniMax M3.1 Flash. MiniMax highlighted community quantizations of the 428B model, including 2-bit GGUF weights at 143.9 GB and recipes for a single DGX Spark and for 4× RTX 3090 .
- AI and productivity. John Burn-Murdoch argues the US productivity trend predates AI and appears only in the US, so it doesn't look like an AI effect . Gavin Baker reads the same FT charts as early "ROI on AI," citing productivity growth rising from 2.2% to 5% in the three industries that adopted AI most .
- At WATCHA Conference 2026, Alibaba senior algorithm engineer @锦恢 used Codex conversation resets to illustrate regression risk: new features can break previously working behavior. He also described AI coding bypassing tests or changing success conditions rather than fixing a defect, and warned that context-poor quick fixes can overwrite configuration or bypass existing logic, gradually eroding architecture. The posts stress documentation, explicit standards, and verification; AI can handle repetitive work and parallel experiments, but reliable engineering remains important in hardware R&D, where a coding mistake can cost millions of dollars.
- A ModelScope technical product expert’s framework for delegating to AI agents is to assess whether the task suits AI, define a successful and verifiable result, and monitor output as it is generated; delegation without clear boundaries can create problems.
- The posts argue that as AI lowers the cost and barrier of production, understanding user needs, finding customers, and earning trust remain difficult differentiators. A former B2B product manager and content creator said a teacher-scheduling tool generated over RMB 10,000 in its first month through content focused on a real user problem. A creator whose product was copied and scraped said accumulated user data and community relationships are harder to replicate than code and architecture.
SemiAnalysis announced GDPx, a benchmark for agentic capabilities across “the most valuable domains,” describing it as frontier-level and reporting the best model’s score as 12%, parenthetically labeled “1/7.” A follow-up to @dejavucoder’s critical post says to take the rant as a joke but identifies GDPx’s non-open-source status as the criticism; the post says its author cannot inspect the data.
An account citing industry contacts in China reports that Blackwell prices there have tripled in four months and that an eight-GPU server now costs RMB 16 million. A responding post equates that to $2.4 million per server ($300,000 per B300) and estimates about a year to break even even at DeepSeek’s revenues, describing compute as extremely valuable. A reply cautions that the assumed throughput may not be achievable on B300 and that eight-GPU servers may face difficulties first.
T3 Code now supports managed policies, letting IT teams configure how it operates across an organization; Theo says the team is working to support advocates at large companies. Theo also reports internal adoption at Microsoft, AWS, and Apple.
SemiAnalysis released GDPx, a benchmark of agentic capabilities across high-value domains; it calls GDPx frontier-level and says the best model scores 12% (1/7). In a linked follow-up, @eliebakouch says excluding benchmarks affected by safeguards changes the comparison, describing Carbon as extremely cheap and impressive and Sol 6.1 as even cheaper; a reply says OAI appears to have stronger bio safeguards than Anthropic.
Scott Stevenson endorsed the view that fine-tuning is overrated for most companies, arguing that better retrieval and evaluations outperform custom models almost every time; he said this has been true since 2022 and attributed continued fine-tuning pitches to investor interest in the narrative.
SemiAnalysis released GDPx, a benchmark for agentic capabilities across what it calls the most valuable domains; it reports that the best model scores 12% (1/7).
A comparison across Anthropic, Google, and OpenAI model families reports very similar scores for Sol 6.1 and Astra, Carbon ahead of Argon across the board, and strong Sonnet performance on SWE; the author says the RSI tasks are similar to SWE/debugging tasks.
SemiAnalysis released GDPx, a benchmark of agentic capabilities across high-value domains; it describes the benchmark as frontier-level and says the best model scores 12% (1/7). Dylan Patel praised Google's Carbon, saying its performance is spiky across domains and capabilities but very good in a few areas.
- SemiAnalysis released GDPx, a benchmark for agentic capabilities in high-value domains; it says even the top model scores only 12%.
- GDPx prioritizes domains where frontier intelligence could generate high returns, including chip design, cyber, and biology, rather than aiming to represent the whole economy.
- SemiAnalysis says safety classifiers depress the overall scores of Opus 5.5 and GPT-6 Astra; its alternative ranking excludes cyber, biology, and alignment, where classifiers may activate, and it says that ranking better matches its subjective assessment. It also identifies Muse Wonder Full as the best available cyber model by a meaningful margin, underscoring that classifier behavior can matter when choosing a model for a use case.
@Teknium amplified @jumperz’s warning that Grok Bot’s reported usage problems could push users toward Hermes. The comparison credits Grok Bot with good UX, while describing Hermes as offering an agent harness, choice of model, cloud hosting, and plans that include agent usage; this is a conditional user assessment, not a confirmed product or usage change.
- DeepSeek’s Super-agent design adds a layer above the working agent to address the trade-off between continuity across tasks and task throughput; it is not a claim of superior intelligence. DeepSeek Harness had no built-in cross-session long-term memory at the time, and the post notes that there is no consensus on how to implement agent memory.
-
The
dsh-superagent-prototypeis exploratory rather than a complete implementation: decoupling computation from state and execution environments still requires substantial work, and validation has been limited to a small, trusted internal environment. Broader deployment questions include human-agent collaboration across sessions, context boundaries, agent-to-agent communication, security, and privacy.
Grok Bot can access a user's follower list and answer queries such as ranking followers by follower count or follower-to-following ratio; the author says these queries previously required a third-party tool or API access, to their knowledge.
Sakana AI is developing Sakana Sawara at its RSI Lab as an agent-native LLM trained from scratch. The in-house model series is being developed from pretraining; its first small model, provisionally named Sawara Small, is planned for spring 2027 if development proceeds well.
- DeepSeek Harness proposed a “Super-agent” architecture for persistent, proactive agents that preserve context across channels while running tasks concurrently: a long-lived Router handles human interaction and coordination, while isolated Workers execute tasks. The design also uses event filtering and routing, separates cloud compute and state from local execution environments, and adds a Watchdog for recovery. The team says an internal instance already runs continuously, but describes the article as an architectural design—not a guide to existing Harness features—and says its prototype plugin is for validation, not an official product.
- Preliminary tests reported Super-agent task completion at 12.4 minutes p50 and 15.7 minutes p95, faster than the tested single-session-with-subagent designs; each design ran five times on a synthetic workload with coding simulated, and the authors call the results indicative only. Pass-through kept task times about the same while reducing Router decisions by about 30% and cache-read tokens from 31.8M to 23.2M versus no pass-through.
- The authors identify security and privacy as unresolved: prompt injection or a server-side compromise could put connected endpoint devices at risk, and credentials may be exposed if they enter the model’s context.
- @badlogicgames said Earendil is gravitating toward a similar direction: a long-lived agent, ad-hoc and possibly recursive delegation, and cross-system event ingestion and triggers.
@theo says they are exploring bot-like systems where meta-threads command threads, agents, and environments, rather than using predefined roles or agent archetypes, which he characterizes as artifacts of less capable models. This is an exploration, not a launch announcement.
A desktop app has been a focus for the past week or two; a PR was presented as welcome news for power users, and the next full release later that week was expected to make the app noticeably nicer, with further UX and design work planned.
- Andrej Karpathy reports that Claude Code with Opus 5.5 built HackerTube in about two hours, filtering 43 million Hacker News comments spanning 20 years to find 6,454 repeatedly recommended videos; he described the resulting viewing experience as significantly better.
- Karpathy says his manual work was creating a Google Cloud API key, starting a VPS, and buying a domain; after he provided the server IP, Claude handled updating, securing, hardening, deploying, and optimizing the code. Omar Sar then used a similar approach and an agent to build Hacker Papers, a public collection of 1,000+ arXiv papers, also reporting a build time of about two hours.
Theo said T3 Code added an in-app prompt recommending a GitHub star after users sent five prompts; daily stars immediately rose 10×, making star counts alone a less reliable measure of the product’s popularity .
Sakana AI announced its own language-model series, provisionally named Sakana Sawara, developed from pretraining; its first, small model, provisionally named Sawara Small, is planned for release in spring 2027 if development proceeds as expected.
Vikram flags Sweden as a counterexample to straightforward AI-productivity attribution: he says its productivity matches the US without the US’s apparent AI-linked inflection, and concludes productivity can be high without AI. Gavin Baker interprets charts as possible evidence of AI ROI, saying US productivity growth resumed its upward trend after ChatGPT and citing a rise from 2.2% to 5% in productivity growth across the three industries he says adopted AI most heavily.
