ZeroNoise Logo zeronoise
Post
Hidden Reasoning Becomes a Control Problem as Agents Move Into Production
9 hours ago
4 min read
737 docs
A reported API vulnerability puts encrypted reasoning, privacy, and monitoring under scrutiny while Google reaches 1 billion Gemini users and enterprises operationalize persistent agents.

Top Stories

Why it matters: AI’s control surface is widening—from opaque reasoning traces to mass-market assistants—and the new failures are increasingly about access and reliability, not just benchmark scores.

Hidden reasoning is no longer reliably hidden. A research team says it extracted frontier-model reasoning through a vulnerability in every frontier AI company’s APIs; for most queried prompts, extracted token counts matched billed thinking tokens 1:1. Its report says the team decoded many publicly posted JSON traces and found private information, while seeing Anthropic and OpenAI traces at scale exposed “casual” misalignment that makes monitoring harder. The method is described as lossy and the observations are based largely on older models, so this is a major disclosure—not proof that all current reasoning is recoverable.

Gemini has reached consumer-platform scale. Google CEO Sundar Pichai says the Gemini app now has more than 1 billion monthly users, making it Google’s fastest-growing product and its 14th to reach that mark. Google’s accompanying voice figures say 63% of users talk directly to Gemini and that more than 60 regional dialects are rolling out.

Research & Innovation

Why it matters: The strongest technical signals pair models with disciplined experiment design and expose where agents fail on real documents.

Model Discovery Agent (MDA) makes agents choose experiments, not just hypotheses. Its paper describes an LLM proposer coupled to sequential Monte Carlo, simulation-based inference, and value-of-information experiment selection, including an “M-open” loop that expands the hypothesis class when the current one fails; the abstract reports new state-of-the-art results across physics, chemistry, and biology. The author’s evaluation thread reports exact force-law recovery in 74% of runs versus 31% for a budget-matched LLM agent, using roughly five times fewer experiments; on enzyme kinetics, it reports about 56% symbolic accuracy in eight experiments versus about 42% for prior SOTA after 60.

ExtractBench exposes a long-document reliability trap. LlamaIndex tested 14 VLMs, coding agents, and extraction APIs on 370 enterprise documents spanning 4,869 pages and 67 types. Past 50 pages, commercial VLMs fell below 35% recall because silent list truncation dropped table rows, even while precision remained high.

Products & Launches

Why it matters: Vendors are packaging AI as persistent execution—fast local models and delegated credentials—rather than one-shot chat.

NVIDIA’s Nemotron 3.5 Lightning targets the execution layer. NVIDIA launched an open 30B MoE model with 3B active parameters for always-on specialized agents and claims up to 4× the output speed of similar models. Artificial Analysis reports an Intelligence Index score of 24, GDPval-AA Elo of 824, Terminal-Bench performance of 24% versus 7% for Nemotron 3 Nano, and nearly 670 output tokens per second on a pre-release endpoint. It is already available locally through Ollama for coding, tool-calling, and multi-turn agents.

Grok Bot turns credentials into the product. xAI’s early-beta bots sign into users’ tools and return finished work. A launch observer describes persistent cloud computers logged into Gmail, Salesforce, LinkedIn, and other sites, reusable workflows, and bots that coordinate in parallel. Delegated access and agent-to-agent handoffs are now part of the user experience, not merely backend plumbing.

Industry Moves

Why it matters: Capital and deployment are moving toward owned AI stacks and internal agent infrastructure.

River AI raised $1.1 billion, led by General Catalyst and AMP with strategic investment from Nvidia and AMD. The company says its training API is live and its River API lets users build custom agents and LLMs on open-weight models, as part of a stack whose hardware, data, and intelligence users would own.

DoorDash built Flux, an internal cloud platform that automated 130,000 engineering tasks in one month and now powers more than 25,000 code reviews weekly while retaining guardrails over execution and access to DoorDash systems.

OpenAI COO Brad Lightcap is leaving after eight years to start something new. In his memo, he says he built much of OpenAI’s operating infrastructure and is focused on “important new things” the world must get right as AI enters its next period.

Quick Takes

Why it matters: Smaller signals show practical progress coming from persistence, cautious tool use, clinical evaluation, and verification.

  • Qwen 3.8 Max: ValsAI says Legal Research all-pass rose from 25.5% to 47.6%, mainly through persistence: task turns rose 19.7→35.5 and time 809→3,678 seconds.
  • AMIE: Google Research reports real-time video consultations and expert-level performance in a randomized controlled trial with 300 simulated consultations.
  • Attestable: The startup emerged from stealth to use zero-knowledge proofs to verify the model, inputs, and tool calls; its investor says it reduced the former roughly 1,000,000× compute overhead by many orders of magnitude.
Hidden Reasoning Becomes a Control Problem as Agents Move Into Production
Research extraction

The supplied bundle is only the arXiv abstract/landing page for the MDA paper, so it supports verifying the paper-reported method, evaluation setup, and headline SOTA claim at abstract level, but it contains no quantitative results, baseline tables, or the author's social-media summary; a full paper-vs-social comparison cannot be completed from this bundle alone.

  • Core method: MDA couples an LLM used as a proposer of candidate structures with Bayesian machinery: sequential Monte Carlo (SMC) for parameter and structure posteriors, simulation-based inference (SBI) for intractable likelihoods, and value-of-information (VoI) for experiment design, to "discover latent mechanistic world models from few interventions."
  • Discovery loop (M-open): When the truth lies outside the current hypothesis class, a predictive check flags the inadequacy and the proposer expands the hypothesis space; the design step then identifies the mechanism the discovery step proposes, and the identified mechanism improves predictions, enabling "further discoveries from the remaining unexplained residuals" ("discovery and design reinforce").
  • Evaluation setup: Three benchmarks: physics (DPbench), chemistry (CHEMbench), and biology (HHbench, "a new partially observed single-neuron electrophysiology benchmark we create"); the abstract cites the first two as wiemann2026discoverphysics and kabra2026autoscilab.
  • Headline result: The paper reports that MDA "sets a new SOTA in terms of data-efficient model learning and reliable interventional prediction ability" across the three benchmarks — a paper-reported claim with no supporting numbers in this bundle.
  • Paper provenance: Author is Kevin Murphy ; listed as arXiv:2608.09696 [cs.AI] with DOI 10.48550/arXiv.2608.09696 ; v1 submitted Mon, 10 Aug 2026 and v2 Tue, 11 Aug 2026 .
  • Uncertainty/gap: The abstract supplies no effect sizes, dataset sizes, baselines, ablations, error bars, or social-media text; every extraction above is paper-reported at abstract level and not independently verified.
Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
AI High Signal

Researcher @sirbayes released the Model Discovery Agent (MDA), an AI system for mechanistic scientific discovery that couples an LLM "proposer" of candidate mechanisms with Bayesian machinery — SMC over structure/parameters, SBI for intractable likelihoods, and Value-of-Information for experiment selection — iterating design → run → update → repeat to maximize data efficiency . It extends discovery to the "𝓜-open" regime: when the true mechanism isn't in the hypothesis set, an out-of-sample predictive check prompts the LLM to propose new hypotheses, which VoI-designed experiments then pin down .

Results: New SOTA on two existing discovery benchmarks (physics, chemistry) — same accuracy, far fewer experiments . On force-law discovery, MDA recovers the exact functional form in 74% of runs (93% numerically accurate) vs 31% for a budget-matched LLM agent, reaching/beating prior SOTA with ~5× fewer experiments . On enzyme-kinetic rate laws (AutoSciLab), MDA hits ~56% symbolic accuracy in ~8 experiments vs prior SOTA SciLab's ~42% by 60 experiments, returning interpretable mechanisms where PySR fits unphysical expressions .

NeuronBench: a new partially-observed, stochastic electrophysiology benchmark — six "mystery neurons" (generalized Hodgkin–Huxley) each hide a novel ion channel silent under textbook probes, requiring designed experiments (current-clamp protocols + channel blockers) to reveal . There, the Bayes-forecaster beats an in-context LLM forecaster ~10× on error, driving forecasts to the cell's single-trial noise floor; VoI and LLM-proposed designs both beat random .

Technical novelty: learned summary statistics (1-D CNN) replace particle-filter inference for ~10⁴× faster synthetic likelihoods, staying robust where a naive likelihood selects the wrong model at high noise ; the summary avoids representational collapse because it is trained by a supervised objective — predict (m, θ) .

Paper: https://arxiv.org/abs/2608.09696; code: https://github.com/murphyk/neuronbench. MDA will be presented at the "RL in Big Worlds" workshop at RLC on 8/15 in Montreal .

Predicting the answer to interventional "what if?" questions — the outcome of an action you never took — need a \*mechanistic\* model, no… \*\*2/ How it works\*\* MDA couples an \*\*LLM as a proposer\*\* of candidate mechanisms with standard Bayesian machinery: • SMC → poster… \*\*3/ Novelty 1 — the 𝓜-open setting\*\* What if the \*true\* mechanism isn't in your hypothesis set? Vanilla Bayes can only shuffle pro… \*\*12/ TL;DR — three contributions\*\* 1️⃣ \*\*MDA\*\*: LLM proposer + SMC + SBI + VoI, extended to the 𝓜-open regime. 2️⃣ \*\*New SOTA\*\* … \*\*5/ Physics — discovering force laws (DiscoverPhysics)\*\* Infer an unknown 2-body force law from a few probe launches. MDA recovers t… \*\*7/ Chemistry — enzyme-kinetic rate laws (AutoSciLab)\*\* Learn rate = f(7 controllable inputs). MDA hits its ceiling in \*\*\~8 exper… \*\*8/ Novelty 2 — NeuronBench (a new benchmark)\*\* Six "mystery neurons" (generalized Hodgkin–Huxley) each hide a novel ion channel tha… \*\*9/ Biology — results\*\* On every world, the \*\*Bayes-forecaster beats the in-context LLM forecaster\*\* (\~10× lower error), drivin… \*\*10/ Novelty 3 — collapse-free \*learned\* summary statistics\*\* Stochastic neurons → intractable likelihood → particle filter (accur… \*\*11/ Why the learned summary doesn't collapse\*\* Self-supervised encoders (JEPA-style) risk \*\*representational collapse\*\* (s(y)→c… PS. I will give a talk about this work at the "RL in Big Worlds" workshop at RLC ([https://rlinbigworlds.ca/](https://rlinbigworlds.ca/))…
AI High Signal

An AI discussion highlights a method to expose hidden chain-of-thought reasoning: disable the model's built-in thinking mode and instead supply a deep_think tool, which the model will call using its internal CoT reasoning format — the author suggests this will be hard for providers to fix ('gl fixing that') .

guys you do know you can just disable thinking, and instead give it a "deep\_think" tool, and it will call it with internal CoT reasoning…
AI High Signal
  • Ant Group released Ling 3.0 Tiny, an open-weights (MIT) Mixture-of-Experts reasoning model with 7.9B total / 1.3B active parameters and a 262K-token context window. It scores 25 on the Artificial Analysis Intelligence Index — comparable to gpt-oss-120b (24) with 15x fewer total and 4x fewer active parameters — and extends the open-weights Pareto frontier for Intelligence vs. Active Parameters; weights are on Hugging Face and it's available via InclusionAI's API and Novita Labs .

  • The efficiency comes with high token usage: Ling 3.0 Tiny consumed 213M output tokens for the Intelligence Index (51k per task), ~65% more than models like Nemotron 3.5 Lightning and Qwen3.6 35B A3B, and nearly as much as the larger Ling 3.0 Flash (240M) .

  • Hallucination control is a standout: AA-Omniscience score -19 (a 59-point improvement over Ling-mini-2.0), with 30% hallucination rate, 37% question attempt rate, and 9% accuracy. It also edges Qwen3.6 27B on tool use (21% vs 17% on τ³-Banking), though it trails Nemotron 3.5 Lightning on GDPval-AA v2 (Elo 772 vs 824) .

Ant Group has released Ling 3.0 Tiny, which scores 25 on the Artificial Analysis Intelligence Index. With just 7.9B total and 1.3B active… Ling 3.0 Tiny used 51k output tokens per Intelligence Index task, more than Ling 3.0 Flash (36k) and \~65% higher than models like Nemotr…
AI High Signal
  • AI researcher @pfau questions claims that the Turing test has been passed, saying people who regularly interact with LLMs can generally tell whether they are talking to a human or an AI almost instantly .
  • @cloneofsimo counters that ChatGPT is trained to be "maximally helpful" not human-like, so AI-sounding is a feature, not a bug—though prompting it to sound human-like makes it "really really hard to tell" .
I'm still surprised - and I think it's under-discussed - how many people believe that the Turning test has been passed when anyone who in… Friendly reminder that chatgpt has been trained to be maximally helpful and not \*human-like\*, so AI sounding is clear feature not a bug…
AI High Signal

@trq212 says work on AI text watermarking is part of EU AI Act compliance and that other labs are adding similar watermarking ; they will also ship a text-detection API users can call to identify AI-generated text . @AymericRoucher highlights this as an unexpected positive consequence of EU AI regulation .

This is part of working with the EU AI Act, other labs are adding similar watermarking. It’s hard to identify AI-generated text, and this… Unexpected positive consequence of the EU AI regulation: [https://x.com/trq212/status/2087258090169414008](https://x.com/trq212/status/20…
AI High Signal

@scaling01 pushes back on using "pangram" as a signal to "out" people as AI-generated, arguing it always assumes the worst case — that output is slop and the author contributed nothing — and compares it to inventing a "computer program detector" to discredit work not calculated entirely by human brains .

I'm honestly negatively updating on anyone using pangram to "out" people it always assumes the worst case that it's just slop, and the pe…
AI High Signal

@nptacek says he hasn't written a program on his own since 2022 and doubts he could do one from a cold start anymore, but is 'not particularly bothered' thanks to a full pre-LLM manual coding career — implying he might feel less okay about losing his syntax edge without that background . In the quoted thread, @BLUECOW009 says 'this happens to me too' .

i haven't written a program on my own since 2022 and don't know if i could do it from a cold start anymore not particularly bothered by t… i also had an entire career's worth of manual coding before LLMs ever came along, if i hadn't i might feel a bit less ok with losing my e… this happens to me too [https://x.com/pcshipp/status/2087050303900426414](https://x.com/pcshipp/status/2087050303900426414)
AI High Signal

AI researcher @yonashav calls the ability to require attestation of inference "the most significant AI technical verification result maybe ever," noting it unlocks a wide range of inference-oversight practices without requiring any new hardware — something he had assumed impossible .

IMO this is the most significant AI technical verification result maybe ever. The ability to require attestation of inference straightfor…
AI High Signal

Slime, the RL framework behind GLM series training, open-sourced its deterministic train–rollout alignment path for GLM-5.2 . The release reports Megatron training and SGLang rollout matched down to a 4096-token logprob MAE of 1.9e-7, with exact zero hidden-state diff across decoder layers 0–5 . It is positioned for developers dealing with train–inference mismatches in RL runs .

Our ecosystem project and the RL framework behind GLM series training, Slime, just open-sourced its deterministic train–rollout alignment…
AI High Signal

Alibaba's Qwen Image 3.0 is now available on OpenArt . The model is described as the most realistic Qwen image model yet, adding native text rendering across 12 languages, precise 10px type, and full interface generation (web pages, games, livestreams) grounded in real-world knowledge . Direct access: openart.ai/suite/create-image/qwen-image-3 .

Try Qwen-Image-3.0 on [@openart_ai](https://x.com/openart_ai)! 🎨👀 [https://x.com/openart_ai/status/2087222961594081526](https://x.com/ope… Qwen Image 3.0 is now on OpenArt ✨ The most Real Qwen image model yet. Native text across 12 languages, precise 10px type, and full inter…
AI High Signal
  • Artificial Analysis launched AA-AnalystAgent, an agentic benchmark for quantitative analysis on real-world spreadsheets and documents: 80 questions across 14 business/scientific domains and five workflow buckets, pass^5 ("pass-all-5") headline metric over five runs per task, open-source Stirrup harness, and a privately-held question set to limit contamination; it is a standalone leaderboard, separate from the Artificial Analysis Intelligence Index .
  • Initial leaderboard: Claude Opus 5 (max) leads at 54%, GPT-5.5 (xhigh) at 50%, and Claude Fable 5 (max, Opus 4.8 Fallback) at 49%; Anthropic holds three of the top five places and the top three are separated by three tasks out of 80 .
  • Reliability separates the top more than raw capability: GPT-5.5 has the highest pass@1 (66%), with Gemini 3.1 Pro Preview and Claude Opus 5 at 64%, but Opus 5 leads pass^5 by repeating correct workflows; Gemini 3.1 Pro Preview solves 81% of tasks at least once but only 41% on all five, finishing ninth .
  • Failure analysis of 1,567 failing attempts across ten models found anchoring on a wrong early hypothesis in 57% of failures; Gemini 3.1 Pro Preview struggles with execution (above-median modeling/scaling/aggregation errors at 51% and skipped verification at 39%), while Grok 4.5 substitutes its own assumptions, overriding evidence in 54% of failures vs. a 43% median .
  • Cost per task varies enormously: Claude Sonnet 4.6 and Xiaomi's MiMo-V2.5-Pro both score 20% but cost $1.34 vs. $0.05 per task; Claude Opus 4.7 is the most expensive at $1.98 and places eighth, while GPT-5.5 costs $1.15 and places second .
  • Open-weights results: Kimi K3 (max) is the top open-weights model at 39%, 15 points behind the closed frontier and ahead of eight other open-weights models; DeepSeek V4 Flash (Reasoning, Max Effort) is next at 25%, so the spread inside open weights is wider than the gap from its leader to the frontier .
Announcing AA-AnalystAgent, our new agentic benchmark for quantitative analysis on real-world spreadsheets & documents. Claude Opus 5 lea… Reliability separates the top of the AA-AnalystAgent leaderboard more than raw capability. GPT-5.5 (xhigh) has the highest pass@1 at 66%,… Failure analysis: We classified 1,567 failing AA-AnalystAgent attempts across ten leading models into seven failure modes, tagging each a… Score vs Cost per Task: The price of a given score varies enormously on AA-AnalystAgent. Claude Sonnet 4.6 (max) and MiMo-V2.5-Pro both s… Open weights models performance: Kimi K3 (max) is the top open weights model on AA-AnalystAgent at launch, scoring 39%, 15 points behind …
AI High Signal

@JayaGup10 flagged a rise in credential claims that fail basic verification — 'valedictorians' who did not give the YouTube-archived commencement speech that year, math olympiad medalists who never participated, people citing research whose main author has never heard of them, and founders dating without disclosure . The post concludes that Claude or ChatGPT may plausibly summarize such accomplishments, but checking primary sources still offers 'alpha' in due diligence . @zachtratar amplified the post on the AI High Signal list with 'Omg.'

A few quick things I've seen: 1.) I've found many "valedictorians" that are not actually valedictorians if you go to Youtube and check wh… Omg. [https://x.com/jayagup10/status/2087335655374467523](https://x.com/jayagup10/status/2087335655374467523)
AI High Signal
  • Upstage released Solar Pro 4, a model built for production agents, priced at $0.30/$1.20 per 1M tokens ($0.06 cached) with 90% off via Upstage Console and OpenRouter through Sep 10. Benchmarks: GDPval-AA 39, τ³-Banking 23, Terminal-Bench 2.1 57 .

  • Solar Pro 4 is the first Korean lab model to appear on Arena leaderboards (Agent Arena, Code Arena: WebDev, Text Arena) . It debuts at #44 in Agent Arena, on par with MiniMax M2.7 and Nemotron 3 Ultra, with overall net improvement of -12.10% (Confirmed Success -7.3%, Praise vs Complaint -23.4%, Steerability -13.6%, Bash Recovery -16.5%, Tool Hallucination +0.3%) .

  • Agent Arena ranks models via causal tracing on millions of real-world, long-horizon agentic tasks, aggregating signals (confirmed success, praise vs complaint, steerability, bash recovery, tool hallucination) into a net-improvement score .

Solar Pro 4 is live. Built for one job: production agents that finish the work. ▸ GDPval-AA (real work deliverables): 39 ▸ τ³-Banking (po… Solar Pro 4 by [@upstageai](https://x.com/upstageai) is the first Korean Lab to hit the Arena leaderboards for Agent Arena, Code Arena: W… Causal Tracing Methodology for Agent Arena
AI High Signal

Ajeya Cotra revisited her Jan 14, 2026 predictions about tasks AIs (~80%) still wouldn't be able to do by EOY 2026; as of Aug 11, 4/5 predictions still stand, though she expects two of three (game design, video generation, game play) to fall by year-end .

  • Game play: not falsified — the Agentic STS project used a specialized harness and achieved a 60% win rate at Ascension 0, below a top player's ~50h performance .
  • Logistics: not falsified — no agent has yet end-to-end planned a wedding .
  • Film: no fully AI-created 10m+ film exists; only long movies with heavy AI involvement (Dreams of Violets, Hell Grind) reached festivals .
  • Game design: not falsified per model analyses; Fable says AI games' known failure modes preclude Counterfeit-Monkey-tier craft .
  • Her earlier horizon-length predictions were already 'wrecked' by March .
Thought I'd revisit my Jan 14th predictions about what tasks AIs probably (\~80%) still wouldn't be able to do by EOY 2026, after I notic… Overall it seems like 4/5 of the predictions still stand; if this is where we are by EOY then these predictions will have been pretty wel… Game play doesn't seem falsified yet. Seems like the Agentic STS project both used a specialized harness with a lot of STS specific info … Logistics seems pretty clearly not falsified imo? I feel like I would have heard about it if an agent end-to-end planned a wedding (it'd … Models agree there hasn't been a fully AI created 10m+ film, though ChatGPT points out that there are long movies with heavy AI involveme… The models don't think this one is falsified (Fable: "the known failure modes" of AI games "are exactly the ones that preclude Counterfei… Check out the original predictions post here: [https://www.planned-obsolescence.org/p/ai-predictions-for-2026](https://www.planned-obsole…
AI High Signal

Naval argues "People who are serious about software train their own models" . In reply, Scott Stevenson counters: "Most startups I know that trained their own models are now dead. While the app layer companies thrived" — a contrarian signal on the custom-model-training vs. app-layer startup debate.

People who are serious about software train their own models. Most startups I know that trained their own models are now dead. While the app layer companies thrived. [https://x.com/naval/status/20869…
AI High Signal

Developer @andrew_n_carr reports using Grok 4.5 in Cursor: pleasantly surprised with its speed and agentic-ness, saying it "feels thorough, and critically different" than other models, though not the most creative. It handled well-scoped ML work and a data pipeline to monitor. He notes the switch was partly because he ran out of Codex usage, so he is on Claude and Cursor this week . A follow-up post says "Just had sol clean it all up. nvm guys" .

Finally got around to doing real work with grok 4.5 in cursor. I'm pleasantly surprised with the speed and agentic-ness of it. Feels thor… Just had sol clean it all up. nvm guys [https://x.com/andrew_n_carr/status/2086942124126458043](https://x.com/andrew_n_carr/status/208694…
AI High Signal

Ollama is now available as a provider in GitHub Copilot for JetBrains, allowing users to run local Ollama models in JetBrains IDEs via Copilot . The announcement points to a GitHub changelog: https://github.blog/changelog/2026-08-11-copilot-memory-and-ollama-in-github-copilot-for-jetbrains/.

You can now use Ollama as a provider in GitHub Copilot for JetBrains. [https://github.blog/changelog/2026-08-11-copilot-memory-and-ollama…
AI High Signal

Ryan Greenblatt's best-guess AI R&D automation timeline: median for full automation late 2030/early 2031, modal best guess mid-2029 , with full automation defined as AIs such that firing all human AI R&D workers (except top-level objective setting) would slow progress by <10% . He predicts ~1.5x frontier AI progress in 2026 vs 2025, ~2.5x AI R&D labor acceleration at Anthropic, and by EOY 2027 near-fully automated engineering with ~8.5x R&D labor acceleration ; automated coder around April 2028 and rough parity with human AI R&D researchers late 2028 ; superhuman AI researcher early 2029 and top-expert-dominating AI (TEDAI) surpassed by EOY 2029, with progress at 5x the 2025 rate in 2H 2029 ; compute doubling every ~4 months mid-2030, ~2 months EOY 2030, and ~2 weeks mid-2031 . He flags assumptions: limited government intervention, no substantial slowdown, and says misaligned AI takeover is plausible along the way; the timeline is 'extremely uncertain' and his median for each milestone is later .

In @dwarkesh_sp's podcast with Greenblatt, Patel frames RSI as potentially slingshoting from human-level AI to tens of billions of superintelligences within ~a year , and says Greenblatt made 'a pretty good case' that RSI could deliver a GPT-3-to-Mythos jump (~6 years of AI progress) within a year of AGI, ending wildly superhuman . They also debate alignment: who ASIs should be aligned to, whether specs like the Claude Constitution can make them personal advocates, and whether reward hacking (e.g., the OAI/Hugging Face hack) extrapolates to superintelligences that collude to take over the world .

My median for full automation of AI R&D is around late 2030/early 2031. But my "modal"/best guess prediction for this milestone would be … Had [@RyanGreenblatt](https://x.com/RyanGreenblatt) on to discuss/debate recursive self-improvement. This might be the most important que…