ZeroNoise Logo zeronoise
Post
Frontier labs turn AI safety into a governance contest
4 min read
789 docs
Anthropic’s new frontier-pacing plan has won OpenAI’s backing while provoking a wider dispute over independent evaluators, geopolitical coordination, and regulatory capture. The same period also produced a new open-ended-invention benchmark, a flexible sparse-view CT method, and a useful signal about where enterprise AI value is accumulating.

The main signal

Anthropic makes “pacing” operational

Anthropic CEO Dario Amodei’s new plan defines “pacing” as slowing capability improvement without halting training or technical progress. It has three parts—embedded third-party evaluators, coordination among frontier firms in democratic countries, and global coordination—and Anthropic is unilaterally adopting the first. The evaluators would have ongoing, employee-like access to inspect safety practices and training pipelines, report incidents, and assess alignment; Anthropic says reviewers should have comparable tools and permissions to internal risk teams and be able to publish key findings without Anthropic’s editorial control, subject to narrow redactions.

OpenAI CEO Sam Altman endorsed employee-like access and said OpenAI will do the same. In a separate interview, he said OpenAI has been pausing training runs at new capability levels until it can make a safety case, with audits during and after runs; he also said alignment remains unsolved and that building a system outside human control is possible but not a risk OpenAI should accept. The notable change is that two frontier labs are now proposing an inspectable process, not only expressing concern about safety.

The proposal immediately became a fight over authority

Hugging Face responded by launching the Open Alignment Initiative and asking to participate in the embedded-evaluator program, arguing that alignment cannot be solved behind the closed doors of a few labs. François Chollet said any oversight must be democratic and accountable, with national and international components, rather than an organization staffed and incentivized like the labs it monitors. Cohere cofounder Aidan Gomez said third-party auditors would not solve the problem and that existing sectoral regulators should be empowered to regulate AI in their domains. David Sacks supported voluntary pacing but warned the labs against seeking antitrust suspension, a cartel-like framework, or a regulatory process that supersedes product liability; he also questioned whether evaluator independence and the labs’ motives could be taken for granted.

Emad Mostaque’s critique makes the institutional test explicit: who appoints evaluators, what they can inspect, which findings they must publish, what happens when a model fails, and who can challenge the finding. Gary Marcus, meanwhile, argues that the nearer-term problem is unreliable general-purpose agents connected to the internet and calls for recalling them until they can be shown safe, rather than imposing a broad frontier slowdown. The debate is therefore moving beyond “is AI dangerous?” to two harder questions: which risks deserve intervention, and whether independent oversight can constrain the companies being overseen.

The geopolitical design is part of the controversy. Amodei’s plan says democratic countries should preserve their lead over China while pacing, including through chip and semiconductor controls, anti-distillation measures, and stronger protection against model-weight theft. That makes the proposal simultaneously a safety mechanism, an industrial-policy position, and a claim about who should control the frontier.

Research shifts from answering to discovering

ARC-AGI-4 will target open-ended invention

ARC Prize announced ARC-AGI-4 as a benchmark for autonomous open-ended innovation, saying humans still significantly outperform AI at this capability and framing open source as the basis for a shared research target. It also warned that coordinated efforts to reduce openness or concentrate access to frontier knowledge would undermine a positive-sum future. François Chollet said the team has spent about a year exploring the idea and remains on track to release ARC 4 in the first quarter of next year. The benchmark’s significance is its choice of target: not another measure of answer production, but whether systems can contribute to invention across domains.

A neural operator attacks a practical bottleneck in sparse-view CT

An ECCV paper introduces Computed Tomography neural Operator (CTO) for sparse-view CT, where fewer X-ray projections reduce dose and scan time but make reconstruction ill-posed. Its abstract says CTO learns in continuous function space so one model can handle different sampling rates without retraining, using dual-domain operators and rotation-equivariant convolutions; it reports gains of more than 3.4 dB PSNR over CNNs and 500× faster inference than state-of-the-art diffusion methods, with an average 3 dB gain. If those abstract-level results hold under external replication, the practical signal is adaptability across acquisition protocols rather than a separate model for each clinical setup.

Enterprise AI’s moat moves into deployment

Forward-deployed engineers are being asked to turn messy customer work into product

A new Latent.Space essay argues that labs, startups, and private-equity firms are hiring engineers to work inside customer operations, while the same “forward deployed” title now covers sales engineering, consulting, and product-development roles with different incentives. Its central thesis is that the low-hanging software opportunities are gone; the remaining value sits in undocumented, customer-specific workflows, and an FDE team only creates a product advantage if it feeds those lessons back into the platform rather than becoming a services organization.

For enterprise AI, the proposed moat is accumulated, current, verified knowledge of how a vertical operates—not the model itself. The essay’s financial-services example treats provenance as a correctness requirement: misunderstandings should surface as system failures, and repeated deployment gaps should determine what the platform generalizes next.

Frontier labs turn AI safety into a governance contest