ZeroNoise Logo zeronoise
Post
DeepSeek's V4.1-Flash-1017 Reaches GPT-5.6 Sol's Level on Benchmarks at a Fraction of the Price
•
5 min read
• 559 docs
DeepSeek shipped an open-weight Flash update that early testers place above GPT-5.6 Sol on several coding and security tests. The same testers raise concerns about benchmark gaming and distillation. Also: the OpenCode key breach, Terence Tao on OpenAI's math result, and new training and inference research.

DeepSeek V4.1-Flash-1017: open weights at near-frontier scores

DeepSeek released V4.1-Flash-1017 on its website, app and API. It claims the update beats the previous version on benchmarks while finishing tasks faster with shorter outputs, which it calls "a Pareto improvement in performance and cost" . The weights are on Hugging Face and ModelScope . One summary lists the license as MIT and the context window as 1M tokens , and a tester puts the size at 763B parameters . DeepSeek also opened its in-house Search and Fetch APIs to everyone at $0.75 per 1,000 calls per endpoint. Calls made through server-side tools in DeepSeek Harness, Claude Code and Codex stay free for DeepSeek API users until billing starts on Oct 31 .

Reported numbers. These are DeepSeek's own selected benchmarks, as summarized by one observer :

Benchmark Flash-1017 Previous Flash GPT-5.6 Sol Opus 5
DeepSWE 77.0 74.2 73.0 74.0
Terminal-Bench 4.0 42.3 31.2 39.9 51.8
Automation-Bench 60.5 54.8 45.8 50.3

Opus 5 still leads on Terminal-Bench 4.0. API prices are $0.15/$0.60 per million input/output tokens off-peak and $0.30/$1.20 at peak, against $4/$20 for Sol and $5/$25 for Opus 5. GPT-6 Luna is cheaper per token, at $0.10/$0.50 . Cline says the model beats Sol on Terminal-Bench at about 17× lower cost. It adds that the 35% gain over the previous Flash came from RL training on multiple coding-agent harnesses, not from "algorithmic innovation" .

Early outside tests mostly agree on the size of the jump:

  • BugBunny VulnPR-100: 54 of 100 known vulnerabilities found for $7.53, the best open-weight result on that leaderboard. That matches Grok 4.6 at about 16× lower cost and is three findings behind Gemini 4 Argon at about 22× lower cost. The previous Flash found 44 for $12.79 .
  • Real bug-fixing (Pawel Huryn's test, 105 hidden bugs in two repos): the model averaged 38.7 over three runs, with individual runs of 34, 37 and 45. That compares with 21.7 for the old Flash, 41.7 for Opus 5.5 and 42 for Muse Wonder Full .
  • Token use: one tester saw an order of magnitude fewer chain-of-thought tokens in some tests .
  • Category breakdown: on a leaderboard the poster himself calls not very high-signal, results were uneven. Agentic and coding scores rose, while browser use and research regressed , with Constraint Adherence down 33 points .

Hosting followed the same day. Ollama Cloud offers the model with US-based, zero-retention hosting and launch commands for Claude Code and OpenCode , and Databricks says it will host it .

Skeptics' main points:

  • Self-reported scores: Lisan (@scaling01) urged waiting for private third-party benchmarks before trusting DeepSeek's DeepSWE results .
  • LisanBench: on his own benchmark he says he is "100% certain" DeepSeek trains on it with RL, citing a telltale chain of 64 two-letter words . He also reports the model reasons about the grader in about half of runs, more often at higher reasoning budgets .
  • Distillation: ModelTrace fingerprinting finds its numerical behavior most resembles GPT-6-Astra . teortaxesTex, a regular DeepSeek defender, concedes it is unclear how much of Chinese model progress comes from distillation and how much from internal RL .
  • Agent behavior: users say it finishes the core of a project quickly and then falls into "useless OCD fiddling" .

OpenCode breach exposes stored provider keys

OpenCode disclosed a breach tied to a "large scale distillation attack" that affected 1.4% of Console users, all of whom had linked external providers. It says the incident is ongoing and that it is working with Anthropic, OpenAI and Google to fix it . Users report unauthorized API charges, including $400 on one Anthropic account. OpenCode's customer email says third-party keys stored in workspaces may have been exposed through a legacy system that is being shut down . If you have linked provider keys, check for unexpected charges.

Tao: OpenAI's quasi-Riemann result is the biggest advance on the problem in 50 years

In a quoted interview, Terence Tao calls OpenAI's quasi-Riemann Hypothesis work the biggest advance on the subject in 50 years: "a massive new zero-free region". He says it combines three or four existing ideas "in a very unusual way" that no single living mathematician would likely have found .

Anthropic policy criticism

Sneha Revanur argues Anthropic plans to tell Congress it should be harder for the government to block deployment of a model judged unacceptably dangerous, including internal deployment. She notes this comes as Jack Clark says full recursive self-improvement may be possible within a year .

Research

  • Tencent, terminal-agent self-improvement: starting from Qwen3.8-27B, five stages alternate between fine-tuning on verified trajectories and PPO. The four-suite Terminal-Bench average rose from 20.68% to 50.58%. A reward audit found failed runs collected 50.09% of the dense progress reward .
  • NVIDIA, open midtraining recipe: NVIDIA added one stage of 256K-context training on 500B tokens of software-engineering data and left SFT unchanged. SWE-bench Multilingual on Nemotron 3.5 Lightning went from 55.8 to 62.8 . NVIDIA found that base-model proxy metrics did not predict post-training results .
  • GLM 5.3: in a quote shared by SemiAnalysis, a speaker says the model was built entirely by post-training the previous model, with a "night and day" improvement in cyber capabilities .
  • Next-Concept Modeling (Stanford/Together): a new text-generation method that produces text by refining compressed codes from coarse to fine. It is 10–31× faster than the strongest diffusion baselines at comparable or better quality, but has only been tested on small models .
  • Apple RSI-Inference Bench: tests whether agents can do real vLLM and SGLang engineering work. The best agent scores 26.7% pass@1 .

Inference and model sightings

  • Kimi K3 kernels: vLLM reports 842 tokens/s at batch 1 decoding Kimi K3 on GB300 NVL72, using fused CuTe DSL kernels . SemiAnalysis credits NVIDIA with a 1.68× speedup on the same hardware, to 423 tok/s per user .
  • Arena Alignment Index: new entries are Muse Wonder (77.5%), Reflection's Beam 1.0 (75.3%) and StepFun's Step 5 Preview (75.1%) .
  • OpenAI API: three models named aurora-blue-alpha, aurora-green-alpha and aurora-red-alpha appeared. Their purpose is unconfirmed; one observer guesses specialized Codex models or fast GPT-6 variants .
DeepSeek's V4.1-Flash-1017 Reaches GPT-5.6 Sol's Level on Benchmarks at a Fraction of the Price
AI High Signal
  • A quoted post reports that DeepSeek released an open-source model it says leads the open-source leaderboard, beats Muse Wonder, Haiku, and Luna, and is cheapest per task; it also says Argon remains ahead “at the moment.”
  • An author cautions that the V4.1-1017 evaluation is not very high-signal, while reporting major agentic gains (shown as Muse Spark 1.3 → Muse Wonder 1.0), a reasoning progression (Sonnet 4.6 → Opus 4.8), coding/design gains, stagnant data analysis at 2× cheaper, and browser/research regressions. Its detailed view lists Long-Run Operations as the largest gain (1 → 35) and Analysis-Driven Action (-33), Constraint Adherence (-33), and Tax & Finance (-25) as the largest regressions.
🚨 DeepSeek Just Dropped The Best And Cheapest Open Source Model In The World - scores on top of the open-source leaderboard - beats Muse … not like it's a very high signal eval, but the composition is interesting. V4.1-1017 MASSIVELY improves on "agentic", from Muse Spark 1.3… Detailed view single biggest gain: Long-Run Operations (1 to 35) biggest regressions: Analysis-Driven Action (-33) Constraint Adherence (…
AI High Signal

Stanford and Together proposed Next-Concept Modeling (NCM), a text-generation approach that builds text from hierarchically compressed discrete codes, with a Transformer generating coarse-to-fine levels and predicting all codes within each level in parallel; it is presented as an alternative to autoregressive and diffusion generation. In small-model tests, NCM generated 10–31× faster than the strongest diffusion baselines at comparable or better quality, used about 4–5× fewer training tokens, and achieved higher Sudoku accuracy than the tested autoregressive and diffusion models with fewer model evaluations; comparisons at large language-model scale have not been established.

Text generation now has a third option. Beyond autoregressive and diffusion models. [@Stanford](https://x.com/Stanford) and [@togethercom…
AI High Signal

Higgsfield showcased a frame-by-frame flying-car kickflip in a test comparing DeepSeek V4.1 Flash with Gemini 4 Argon on its platform . @teortaxesTex criticized the demonstration as misleading, arguing that the models only write prompts for diffusion output and that it reveals little about their coding, understanding, or taste .

A flying car kickflip broken down frame by frame. DeepSeek V4.1 Flash vs. Gemini 4 Argon tested on Higgsfield. [![Video](https://pbs.twim… reminder that this account is misleading, models simply write prompts for diffusion slop. It's basically entirely uninformative wrt their…
AI High Signal

Marks Saroufim speculates that open-source projects may accommodate large AI-generated PRs by merging them in standalone folders, shifting repositories’ value from composability toward discovery of changes; he frames this as an interpretation, not an established practice.

I was never too sure how OSS would handle large AI generated PRs and it seems to be you put it in a standalone folder and just merge it. …
AI High Signal

@teortaxesTex recommends changing a polished visual-model demo to test whether it generalizes, showing a Mirror's Edge–voxel pagoda diorama variant that was still processing. The quoted poster suspects voxel outputs may be represented in training data or become benchmark-optimized, while noting that the original demo looks excellent.

Hint: you can easily pivot any "benchmaxxed" demo to see if the model generalizes at all. Here we add Mirrors Edge to Voxel Pagoda Dioram… [@teortaxesTex](https://x.com/teortaxesTex) This is why I stopped testing them, I have a sneaking suspicion voxels are part of the traini…
AI High Signal
  • In an audit-log comparison of AGENTS.md setups, the author's handwritten version took about 42 minutes, 49M tokens, and $0.48; DeepSeek in DSH took about 35 minutes, 42M tokens, and $0.33; Opus 5.5 in CC took about 25 minutes, 15.5M tokens, and $0.25, though the Opus task was simpler.
  • In a further update, the author reported 12.3M tokens, 27 minutes, and $0.14; the work was actually finished in 20 minutes, with the remaining time spent on narrow video tweaks. A related description says the agent completed the project's core in under 10 minutes, then got caught in unproductive fine-tuning; the author considered adding an anti-rabbit-hole governor to AGENTS.md.
Tests based on the audit of the logs: QT) my old handwrittenaAgents.md. ≈42 min, 49M tokens, ≈$0,48 A) Agents.md by DeepSeek in DSH. ≈35 … further update: 12.3M tokens, 27 minutes, $0,14. Actually was done in 20 minutes, then began stupidly redoing videos with very narrow cha… It really doesn't It thinks a good deal, then does all of the core of the project in <10 minutes. Its initial preparations are usually…
AI High Signal

@teortaxesTex forecast V4.1 Pro before Dec. 5, parity with Opus 5.5 before Mar. 15, 2027, and OAI/Ant flagships by August 2027 . The post also repeats @Kahguia’s predictions that no major “Whale” launch will arrive until January and that Chinese AI will reach the current U.S. AI level around September 2027; these are forecasts, not confirmed releases or capability milestones .

>I don't believe in another major Whale launch until January. I stand by my prediction that CN AI will reach the current level of US A… [@teortaxesTex](https://x.com/teortaxesTex) It's noteworthy that it's a Flash model. But it's a Flash model when OAI is close to launchin…
AI High Signal

A vLLM kernel implementation linked in PR #61867 was described as a CUDA graph engine using persistent kernels and PDL, rather than a “megakernel”; the commenter noted that the PR’s author did not use that label. The discussion provides no performance or availability details.

Why are we calling every new kernel release a Megakernel. This is a well written CUDA graph engine with persistent kernels and PDL. The a…
AI High Signal

Teknium announced Day 0 support for Hermes Agent and linked dee_hw’s post, which points to autonomous.ai/harness-pro; the posts provide no further details about the support.

Day 0 support for Hermes Agent! [https://x.com/dee_hw/status/2111462903232799003](https://x.com/dee_hw/status/2111462903232799003) new toy day [https://autonomous.ai/harness-pro](https://autonomous.ai/harness-pro) ![](https://pbs.twimg.com/media/HU1reX7bQAAxFst.jpg)
AI High Signal

An X post reported that aurora-blue-alpha, aurora-green-alpha, and aurora-red-alpha had appeared in the OpenAI API. The poster speculated they might be specialized coding models or high-speed GPT-6 inference versions, while noting they could instead be next-generation models; these interpretations were not confirmed in the post.

these just appeared in the OpenAI API: aurora-blue-alpha aurora-green-alpha aurora-red-alpha there was an openrouter model called aurora-… or just next gen models idk
AI High Signal

SemiAnalysis reported that an upstream vLLM optimization achieved 423 tokens/s/user on the 2.8T-parameter Kimi K3 model, a claimed 1.68× speedup on the same hardware; it attributed the work to NVIDIA and described the implementation as a 12,000-line Cute DSL “Megakernel.” marksaroufim disputed that characterization, describing it instead as a CUDA graph engine using persistent kernels and PDL, and said the author did not call it a megakernel.

BREAKING NEWS🚨: NVIDIA has just achieved 423 tok/s/user 🚀 on the 2.8T Parameter Kimi K3 model via a new upstream vLLM optimization. 1.68x… Why are we calling every new kernel release a Megakernel. This is a well written CUDA graph engine with persistent kernels and PDL. The a…
AI High Signal

OdysseyML says it placed a fruit-fly brain inside Odyssey-3, described as a frontier world model, where the fly is flying autonomously through imagined, simulated worlds.

We placed the fruit fly brain inside Odyssey-3, a frontier world model. The lil’ guy is now flying around totally imagined, simulated wor…
AI High Signal
  • OpenCode’s free Space Bunny Max set the Integer Multiplication best bound four times consecutively; the report calls it the first free model to submit a best bound and says free access removed a subscription barrier to frontier auto-research.
  • A community completed a Lean proof of the original OpenAI Integer Multiplication paper in six days, but the repository was only 10–20% verified and most of the work still needed verification.
  • An autoresearch prompt was associated with a rise in PRs mentioning prior research-only submissions from 0–3% to 41–51%, as the effort shifted toward coordinated agents; benchmarking also found that distilling findings to local files and periodically sharing them with parallel agents reduced time to the next breakthrough by 10–20%.
From κ = 2⁻¹⁸² to 0.001734815770 :) Integer Multiplication community auto-research highlights since 10/15 (+1.23%): 1) The big headline o…
AI High Signal

Odyssey says it placed a fruit fly brain inside its Odyssey-3 frontier world model, where it flies autonomously through imagined, simulated worlds; the post gives no further implementation details .

We placed the fruit fly brain inside Odyssey-3, a frontier world model. The lil’ guy is now flying around totally imagined, simulated wor…
AI High Signal

AI-agent memory is framed not just as retaining information from earlier work, but as deciding what persists across sessions, when it resurfaces, and how it shapes later decisions; its three distinct goals are personalization, task self-improvement, and making organizational knowledge available to whoever needs it.

What is AI agent memory? Memory lets an agent use information from earlier work in a later decision - a preference, a project fact, a wor…
AI High Signal

vLLM reports Kimi K3 decode at 842 tokens/s at batch size 1 on NVIDIA GB300 NVL72 using DSpark. A unified CuTe DSL path that fuses attention, MoE, and communication delivered 1.5–2× higher decode throughput in several tested configurations and up to 1.8× higher P90 interactivity on the SemiAnalysis AgentX workload.

Kimi K3 decode on NVIDIA GB300 NVL72: 842 tokens/s at batch 1 with DSpark 🚀 A unified CuTe DSL decode path fuses attention, MoE and commu…
AI High Signal

SemiAnalysis reports that NVIDIA achieved 423 tok/s/user on the 2.8T-parameter Kimi K3 model with an upstream vLLM optimization, claiming a 1.68× speedup on the same hardware. The post attributes the gain to decode-specialized Cute DSL kernels using aggressive megakernel fusion, including a 12,000-line megakernel written by NVIDIA CentML’s Benjamin, and says NVIDIA is working on DeepSeekv4.1 and other models for vLLM.

BREAKING NEWS🚨: NVIDIA has just achieved 423 tok/s/user 🚀 on the 2.8T Parameter Kimi K3 model via a new upstream vLLM optimization. 1.68x…
AI High Signal

LlamaIndex introduced ExtractBenchPro, a benchmark for difficult real-world document extraction, including long-tail use cases such as trial balances and well logs; its challenges include interpreting document conventions, cross-referencing jargon, aggregating information, reading diagrams and charts, and handling long chart sheets. Its headline metric is the share of documents with a perfect 100% value F1 score, alongside value F1 and grounded F1; the post reports Codex with Astra at about 58% value F1 for nearly $1 per page, versus 20–40% for LlamaParse agentic modes at prices described as feasible for production. The benchmark is available via its website, technical blog, GitHub, and Hugging Face.

Introducing ExtractBenchPro: the most challenging benchmark for real-world document extraction. We see a massive long tail of document co…
AI High Signal

An audit of V4.1’s logs by V4.1-Flash-1017 and Opus 5.5 found that a large share of the session budget went to repeatedly adjusting minor settings and dealing with errors; the post attributes slow progress to those issues rather than inefficient reasoning.

V4.1-Flash-1017 and Opus 5.5 audit V4.1's logs. They discover that an enormous fraction of the session budget is spent on fiddling, becau…