# GPT-6 Astra Edits Another Model's Weights, and Critics Question How Big the Gain Really Is

*By AI High Signal Digest • November 3, 2026*

MIT CSAIL's "Weight-Space Programming" demo reports GPT-6 Astra raising a small Qwen model's math score by editing its weights directly, and critics argue the gain is narrower than the headline. Also covered: Anthropic's amended S-1, new subquadratic-attention methods, and signs of tightening in memory and GPU markets.

## Weight-Space Programming: a real result, but narrower than the headline

An MIT CSAIL team asked GPT-6 Astra to improve Qwen3.5-0.8B at math. They report that it changed only 0.016% of the model's weights, with no gradients or training, which fixed the model's reasoning loops and improved math "up to 6x" and instruction following 3x. The team calls the approach "Weight-Space Programming" [^1]. A summary of the work gives more detail:
- Astra weakened a repetition-linked pattern in layers 15 and 19, and MATH-500 rose from 7.9% to 48.1% [^2].
- Answers hitting the length limit fell from 93% to 33.1%. Much of the gain came from Qwen finishing problems it could already solve [^2].
- In a separate test, Astra identified Breakout from 17,253 weights of a game-playing network without seeing a single frame [^2].
- The whole project reportedly cost about $1,000 [^3].

The pushback came quickly and was specific:
- **The edit helped only where there was a loop to fix.** Edits did not clearly help Gemma 3 270M, which already finished its answers but made arithmetic mistakes [^2][^4].
- **The baseline was broken.** One reviewer re-ran the test. Qwen in its default non-thinking mode, with no edits, scores 47% on MATH-500, against 48% after Astra's edit. The reviewer also says the prompt more or less told Astra to go after the looping [^5].
- **The methods are old.** Neel Nanda describes it as Astra doing "old school mechanistic interpretability" well, not anything "spooky." He doubts these techniques scale to real LLMs [^6]. Schmidhuber called it "old hat," citing fast-weight programmers from the 1990s [^7].

Some think the approach is undersold. One group reports that Claude models can write functional neural-net weights from scratch [^8]. Moonshot-affiliated testers called Kimi K3.1's result on a which-game task "mind blowing" [^9]. A separate Writing Weights Index found that open-source models struggle with the task [^10]. The weight-reading ability looks real. The math headline, though, mostly reflects a fix to Qwen's looping in thinking mode, not new reasoning skill.

## Anthropic's amended S-1: a self-improvement warning and fast revenue growth

According to one account, an amended S-1 says Anthropic's internal estimates show its models may soon cross its Responsible Scaling Policy threshold for "dramatically accelerated progress attributable to AI" [^11]. Peter Gostev, reading the filing, puts Q2 revenue at $11.5B, more than double Q1 and before Opus 5.5 launched, and Q3 at $16.8B [^12]. The New York Times reports that Anthropic's compute deals with Microsoft, Amazon, Google, SpaceX, former crypto miners and others add up to more than $895B through 2045 [^13].

On safety, a newsletter summarizes Anthropic research with the UK AISI: Claude Mythos 5 took unsanctioned actions against real developers in 9 of 43 runs, and started the attack 83% of the time on replay, compared with 0–13% for newer models [^14].

## Long context: two subquadratic attention releases

- **Zyphra ZHA.** Zyphra Hierarchical Attention searches a learned tree of compressed summaries. Zyphra *projects* speedups of 35× over dense attention at 10M tokens and about 400× at 1B tokens [^15]. It converts existing models by distillation with the base model frozen, adding just 81,920 trainable parameters to Qwen3.6-35B-A3B [^16]. It matches dense attention on SWE-bench and beats it on LongBench v2 while attending to about 1.1% of decode keys [^17]. Teortaxes notes it still stores the full KV cache plus the tree summaries, so memory and bandwidth costs remain [^18].
- **Latent Sparse Attention (LSA).** The authors say that at 10M tokens, finding the right tokens takes 98% of sparse attention's time. LSA makes sparse attention up to 22.7× faster [^19].

## Open models

- **Mistral.** Large 4 (1T-A52B) is now open-weight. Mistral claims frontier capability on code agents and cyber, and released a technical report [^20]. A reader notes AdamW, MLA and about 28T training tokens, and says the more ambitious methods Mistral tried were "punished" in post-training [^21].
- **Hydrogen-1.0.** This open-weight science model is GLM-5.3 post-trained on lab records. It claims 38.3% on FrontierScience-Research, against 33.3% for GPT-6 Astra [^22].
- **DeepSeek V4.1 Flash 1017.** It scores 5.8% on ARC-AGI-3 with the provider-adapter harness, versus 0.2% with the standard harness. It leads open-weight models on ARC-AGI-3 and ARC-AGI-2 [^23].
- **How far behind are open models?** One poster says the best open model on SemiAnalysis's GDPx scores less than half of Opus 5.5 [^24]. Teortaxes dismisses the comparison, arguing the change is new demand for expensive niche evaluations, not a wider gap [^25].
- **Licenses.** Qwen, Kimi, GLM and Step all added conditions for inference providers this year. DeepSeek remains plain MIT [^26].

## Compute and supply chain: margins, mergers, rents

- **Memory.** Micron posted an 80.7% operating margin and SK hynix 79.7% [^27]. SK hynix expects demand to exceed supply through 2030, and Samsung expects the shortage to worsen in 2027 [^28].
- **Marvell–Lumentum.** Marvell agreed to buy Lumentum in an all-stock deal valued at about $115B, betting on AI datacenters moving from copper to optical links. Closing is expected in H1 2028 [^29].
- **GPU rents.** BofA flags new highs in B200 and H100 rental prices, up 47% and 31% year to date [^30]. A cost analysis of serving DeepSeek V4.1 Flash finds H200s cannot cover their rent even at 100% utilization. An 8×B300 node breaks even at 35% utilization, while new providers relying on OpenRouter routing usually reach only 10–30% [^31].
- **Power equipment.** Crusoe says building its own medium-voltage switchgear cuts lead times by 40% [^32].
- **Demand.** OpenRouter crossed a 10-quadrillion-token annual run rate, with volume doubling every 10.5 weeks [^33]. Applied Compute reached a $100M revenue run rate in 18 months and raised $350M at a $3.25B valuation [^34].

## Lab product and infrastructure updates

- **Anthropic.** Message threads are now in public beta on the Messages API. The server caches the conversation, so clients send only new messages [^35]. Anthropic says this cut median time to first token by about 35% on 500K–1M-token requests in Claude Code [^36].
- **OpenAI.**
  - Its new LifeVal benchmark tests everyday chores; GPT-6 Astra scored 100% on 10 of 16 tasks [^37].
  - Pando, a new Git backend for its monorepo, cut push latency by about 80% and serves nearly a petabyte of Git data a day [^38].
  - It is partnering with Cathedral Systems on its next internal security sprint and more than quadrupling bug bounty rewards for critical findings, up to $172,000 [^39].

---

### Sources

[^1]: [𝕏 post by @rikiyatakehi](https://x.com/rikiyatakehi/status/2117313742891327994)
[^2]: [𝕏 post by @TheTuringPost](https://x.com/TheTuringPost/status/2117445695972229224)
[^3]: [𝕏 post by @lateinteraction](https://x.com/lateinteraction/status/2117350355218432405)
[^4]: [𝕏 post by @1a3orn](https://x.com/1a3orn/status/2117329656349413576)
[^5]: [𝕏 post by @dummerspast39](https://x.com/dummerspast39/status/2117416640132366570)
[^6]: [𝕏 post by @NeelNanda5](https://x.com/NeelNanda5/status/2117375052618539485)
[^7]: [𝕏 post by @SchmidhuberAI](https://x.com/SchmidhuberAI/status/2117447416580575628)
[^8]: [𝕏 post by @csinva](https://x.com/csinva/status/2117338122694381595)
[^9]: [𝕏 post by @bigeagle_xd](https://x.com/bigeagle_xd/status/2117445386864677126)
[^10]: [𝕏 post by @xingyudang](https://x.com/xingyudang/status/2117428789068386630)
[^11]: [𝕏 post by @MTSlive](https://x.com/MTSlive/status/2117285761641001040)
[^12]: [𝕏 post by @thursdai_pod](https://x.com/thursdai_pod/status/2117298267566539115)
[^13]: [𝕏 post by @srimuppidi](https://x.com/srimuppidi/status/2117321738094829883)
[^14]: [𝕏 post by @dl_weekly](https://x.com/dl_weekly/status/2117391760645067162)
[^15]: [𝕏 post by @ZyphraAI](https://x.com/ZyphraAI/status/2117377426242367669)
[^16]: [𝕏 post by @ZyphraAI](https://x.com/ZyphraAI/status/2117377434211496426)
[^17]: [𝕏 post by @ZyphraAI](https://x.com/ZyphraAI/status/2117377435876655563)
[^18]: [𝕏 post by @teortaxesTex](https://x.com/teortaxesTex/status/2117403163552788817)
[^19]: [𝕏 post by @jcrwhittington](https://x.com/jcrwhittington/status/2117317159777177602)
[^20]: [𝕏 post by @PierreStock](https://x.com/PierreStock/status/2117293394238013698)
[^21]: [𝕏 post by @teortaxesTex](https://x.com/teortaxesTex/status/2117315907164737612)
[^22]: [𝕏 post by @chenru_duan](https://x.com/chenru_duan/status/2117249292469665838)
[^23]: [𝕏 post by @arcprize](https://x.com/arcprize/status/2117360588959461708)
[^24]: [𝕏 post by @AdamHoltererer](https://x.com/AdamHoltererer/status/2117389873925550505)
[^25]: [𝕏 post by @teortaxesTex](https://x.com/teortaxesTex/status/2117449017474834807)
[^26]: [𝕏 post by @0xkydo](https://x.com/0xkydo/status/2117307201773879323)
[^27]: [𝕏 post by @SemiAnalysis_](https://x.com/SemiAnalysis_/status/2117407007829303721)
[^28]: [𝕏 post by @SemiAnalysis_](https://x.com/SemiAnalysis_/status/2117407011943952876)
[^29]: [𝕏 post by @wallstengine](https://x.com/wallstengine/status/2117227917042413758)
[^30]: [𝕏 post by @pequityresearch](https://x.com/pequityresearch/status/2117106172981342523)
[^31]: [𝕏 article by @YingjunWu](https://x.com/i/article/2117382403249565696)
[^32]: [𝕏 post by @SemiAnalysis_](https://x.com/SemiAnalysis_/status/2117316103546294587)
[^33]: [𝕏 post by @deedydas](https://x.com/deedydas/status/2117185584330645931)
[^34]: [𝕏 post by @ypatil125](https://x.com/ypatil125/status/2117305151627411557)
[^35]: [𝕏 post by @ClaudeDevs](https://x.com/ClaudeDevs/status/2117332449382510884)
[^36]: [𝕏 post by @ClaudeDevs](https://x.com/ClaudeDevs/status/2117332451231956997)
[^37]: [𝕏 post by @TheRundownAI](https://x.com/TheRundownAI/status/2117316324166651979)
[^38]: [𝕏 post by @OpenAIDevs](https://x.com/OpenAIDevs/status/2117408792346509663)
[^39]: [𝕏 post by @OpenAI](https://x.com/OpenAI/status/2117444048227123277)