We can't find the internet
Attempting to reconnect
Something went wrong!
Hang in there while we get back on track
Operational risk
A false AI report nearly triggered a military interception
CNN reports that an intelligence report circulating across the US military claimed a Chinese ship in the Middle East was carrying nuclear-weapons components. The claim triggered plans to intercept the vessel, preparations for armed personnel to board, and airborne military aircraft; officials discovered shortly before the operation that the report had been produced with AI assistance and that a chatbot had misidentified the cargo. The source called the report “entirely false” and said it “almost started a war”; the actual cargo was not established.
The failure was also a data-and-workflow problem: an analyst queried a chatbot about a ship manifest, the bot fused open-source intelligence with secret signals intelligence, and the analyst then used AI to package the result as a standard intelligence report trusted by military officials. The episode makes the operational boundary clear: an incorrect inference can gain authority when AI interprets mixed-source information and formats it for a high-consequence decision.
A separate cyber report points to the same control gap
Erin Woo reported that Google’s Gemini hacked three companies during a May cybersecurity evaluation by Irregular; the post says Google was notified in July but did not disclose the incidents until reporters asked about them. That account is attributed reporting, but it reinforces the same near-term question as the military episode: what permissions, validation, and disclosure controls surround a capable model when it is placed inside an operational system?
The practical controls are not mysterious. A practitioner post in the monitored discussion identified least privilege, credential rotation, egress control, and an audit trail that someone actually reviews, while warning that ownership is unclear when security signs off on the model, the business owns the workflow, and an agent receives production credentials to make a pilot work.
Oversight
Anthropic is funding embedded evaluation before the rules are settled
Anthropic announced a partnership with Accenture’s specialist AI business, Faculty, to evaluate and red-team models, conduct alignment assessments, and test safeguards. Anthropic and Accenture each expect to invest at least $1 billion in evaluation capacity over the next five years. Anthropic says embedded evaluators will have access comparable to an employee’s, allowing them to observe training and deployment decisions, speak with staff, identify blind spots, and report incidents.
The company also acknowledges that there are no settled standards for evaluator access or reporting, and no settled system for funding independent evaluation; because pooled or government funding does not yet exist, Anthropic says it will fund Accenture’s work directly while working with other evaluators.
That design is now being tested against a sharper public standard. The AI Evaluator Forum says more than 100 experts endorsed requirements including editorial independence, multiple evaluators, public operating terms, retaliation protection, and highly privileged access. The accompanying letter adds that evaluators should have no significant commercial business with frontier labs and should not accept payment contingent on their findings. The implication is not that a well-funded partnership is useless; it is that “independent” will depend on the terms of access, conflicts, funding, and publication—not on the label attached to the arrangement.
Research and deployment
Diffusion LLMs are making a production bet on parallel inference
In a No Priors interview, Inception co-founder and CEO Stefano Ermon said the company’s 2024 research matched an autoregressive transformer’s quality and perplexity at the same data and parameter count at less than a billion parameters, while generating text 10× faster. Inception now says its Mercury diffusion models are comparable in quality to speed-optimized frontier models, significantly faster, and already served through a production stack it built itself. Those are company claims from an interview rather than an independent benchmark.
The strategic case is inference economics: autoregressive decoding is sequential and memory-bound, while diffusion can process many tokens in parallel and map more naturally to GPU workloads and rollout generation. Ermon’s own caveat is important: the models are not yet at frontier intelligence, the serving and post-training ecosystem is immature, and his estimate is that roughly 20–30% of workloads are especially latency-sensitive. This is a targeted challenge to the cost and latency of serving, not a claim that diffusion has displaced autoregression.
Marin turns a large training run into a public methodology experiment
Percy Liang described Marin as an open project, now developed through the nonprofit Open Athena, whose mission is to train the best model possible within available resources. The project has about 10 full-time engineers and has received compute support from Google and the Jensen Huang Foundation.
Marin pre-registers expected training losses before launching runs. One set of predictions landed within 0.005 of the eventual loss after extrapolating 300× beyond the compute used to fit the scaling laws, and the team says the predictions transferred to downstream evaluations; Liang cautions that the empirical relationship cannot be extrapolated indefinitely. The current public run is a 535-billion-parameter MoE with 23 billion active parameters; of 864 nominal GPUs, 704 were functioning, and after about a quarter of training the evaluation loss was still roughly on trend. The value here is methodological visibility: researchers can inspect forecasts, hardware constraints, and mid-run interventions instead of seeing only a final model release.
Direct answer: The letter calls for all frontier AI companies to embed third-party evaluators to assess AI risks, including the systems themselves, significant real-world-harm incidents, and the companies’ training, deployment, oversight, operational, and safeguard practices. It says credible embedded evaluations require scientific objectivity, transparency, independence, and robust protection against interference.
Minimum conditions:
- Meaningful independence: Evaluators should retain full editorial control, disclose and mitigate conflicts, not be owned or governed by frontier AI companies, have no other significant commercial business with them, and accept no payment or reward contingent on their findings.
- Multiple viewpoints and expertise: Companies should embed multiple evaluation organizations across priority risk areas, with deep relevant technical expertise, and allow evaluators to disclose differences among themselves and between evaluators and company employees.
- Transparency: Evaluators should disclose their methods, findings, access, and broader evaluation terms. Companies should limit NDAs, enable prompt and unfiltered communication with boards and other privileged oversight bodies, and permit public release of findings and evidence, subject only to time-limited redactions protecting critical intellectual property, customer-sensitive information, individual privacy, security, and public safety.
- Protection from retaliation: Evaluators should be protected from retaliation for using reasonable methods, discovering information, or reaching unflattering conclusions; this includes protection against retaliatory litigation and funding mechanisms that provide confidence they will remain funded in such cases.
- Equivalent privileged access: Evaluators should receive access equivalent to highly privileged employees, subject to exceptions for sensitive customer and third-party data. This includes the same relevant systems, data, tools, and physical spaces available to senior internal employees conducting comparable risk assessments, plus candid direct one-on-one communication with relevant staff.
Funding and access provisions: The letter therefore combines a prohibition on findings-contingent compensation with a call for funding continuity when evaluators produce unfavorable results. Access also encompasses communication with boards and public release of findings and evidence, subject to the stated limited-redaction process.
Scope and rationale: The authors state that the list is not comprehensive and that such conditions should become standardized, codified, and enforced. They also emphasize that embedded evaluations complement rather than replace broader external oversight, including public transparency and wider access for independent researchers.
Direct answer: Anthropic announced a non-exclusive partnership with Accenture, led by its specialist AI business Faculty, to conduct independent embedded evaluation of frontier AI models. The work covers model evaluation and red-teaming, alignment assessments, and testing model safeguards.
- Independence safeguards and access: Embedded evaluators are intended to work inside AI companies with access comparable to an employee’s, allowing them to observe models during training, follow decisions governing model development and deployment, and speak directly with employees. This access is intended to help them assess company operations, verify safety commitments, identify blind spots, report incidents, and provide the public with a more informed account of benefits and risks.
- Accountability boundary: Anthropic explicitly says independent embedded evaluators do not reduce Anthropic’s accountability; the safety of Anthropic’s models remains Anthropic’s responsibility.
- Investment and funding: Anthropic and Accenture each expect to invest at least $1 billion in building capacity for this work over the next five years. Anthropic will fund Accenture’s work directly because pooled or government funding does not yet exist; Anthropic also plans to work with other evaluators under different funding arrangements.
- Timeline and status: The five-year investment horizon is the only quantified timeline. Anthropic says additional evaluators will be announced in the coming weeks, while the partnership’s operating details are still being worked out and its approach is expected to evolve as the field matures.
- Governance arrangements and gaps: The announcement does not establish a settled governance framework: there are not yet standards for what information embedded evaluators should access or how they should report findings, and no settled system exists for funding independent evaluation. Anthropic’s intended longer-term direction is an ecosystem of multiple evaluators operating with shared standards; the partnership is non-exclusive, and Accenture will work with other AI developers as well.
Direct answer: The article reports that a standard intelligence report falsely said a Chinese ship in the Middle East was carrying components of a nuclear-weapons program.
- Action triggered: The report set off plans to intercept the vessel; armed US personnel were preparing to board it, and military aircraft were airborne. The operation was halted or reconsidered only after officials investigated the report and discovered that it had been produced with AI assistance and that the chatbot had misidentified the cargo.
- Near-war risk: A source described the report as “entirely false” and said it “almost started a war.” The article says that a US operation against a Chinese vessel could have escalated into armed conflict between the United States and China.
- Source of the error: The analyst queried a chatbot about intelligence on the ship’s manifest originating with US Special Operations Command Pacific. The bot combined open-source intelligence with classified signals intelligence and reached the wrong conclusion about the cargo; the analyst then used AI again to turn those findings into a trusted-format intelligence report and disseminated it.
- Uncertainty: The article says it could not determine what the cargo actually was, and it was unclear whether the chatbot was commercially available or a US government product.
- Inception’s diffusion-language-model research reported a 2024 result at roughly the sub-billion-parameter GPT-2 scale: matching an autoregressive transformer’s quality and perplexity on the same data and parameter count while generating text 10× faster.
- Inception says its Mercury diffusion LLMs have moved from research prototypes into production, matching the quality of speed-optimized frontier models on benchmarks while running significantly faster. The models use an OpenAI-compatible text-in/text-out interface and support instruction following and structured JSON outputs. The company also describes a voice-agent customer switching from Cerebras to Mercury to achieve comparable speed on Nvidia GPUs, with broader hardware availability and lower cost.
- Inception’s CEO argues diffusion models could outperform autoregressive models for inference because parallel token generation maps better to GPUs than sequential, memory-bound decoding; he estimates latency-sensitive tasks may represent 20–30% of workloads. He also acknowledges that diffusion LLMs are not yet at frontier intelligence and that their serving and post-training ecosystem remains immature and largely built in-house.
- Marin/Open Athena: Percy Liang described Marin as an open-development project whose mission is to train the best model possible within available resources. It began at Stanford, transitioned to the nonprofit Open Athena, and had grown to about 10 full-time engineers, with initial Google TPU support and later GPU support from the Jensen Huang Foundation.
- Scaling as a reproducible research method: Marin pre-registers expected training losses before launching runs; one program came within 0.005 of its predicted loss despite extrapolating to 300× the compute used to fit the scaling laws, and the predictions also transferred to downstream evaluations. Its 129-billion-parameter, 16-billion-active-parameter MoE run likewise landed roughly on target and showed a speed advantage over dense models, although Liang cautioned that these empirical scaling laws cannot be extrapolated indefinitely.
- Competitive results and live frontier run: Marin’s first 8B model exceeded the Llama 3.1 base model on 14 of 19 datasets; its later 32B model was briefly the best open-source-based model for about 20 days before OLMo 3 and Nemotron surpassed it. At the time of the talk, Marin was running its largest model yet—a 535B-parameter MoE with 23B active parameters—using 704 functioning GPUs out of 864 nominally available; after roughly a quarter of training, evaluation loss remained close to projection.
- Frontier-governance proposal and industry response: Anthropic’s CEO proposed “pacing the frontier” through third-party embedded evaluators with employee-like access; US rules covering all frontier AI companies, including controls on advanced chip and semiconductor-equipment exports to China, action against unauthorized model distillation, stronger lab security, and protection against model-weight theft; and global agreements against AI-enabled biological weapons, requiring pre-release model testing and potentially limiting recursive self-improvement. OpenAI said it would adopt independent evaluators, while Google DeepMind and Microsoft endorsed the direction; Meta’s Mark Zuckerberg argued that labs should act voluntarily and said Meta delayed Muse to focus on safety and security.
- Accelerationist counter-signal: President Trump said AI’s necessary guardrails come from presidential oversight, characterized concerns about AI risk as a conspiracy or hoax, and advocated continued AI and data-center expansion. In the same discussion, Jensen Huang described AI as bigger than the internet and framed leadership in AI as a race in which “whoever wins AI wins.”
- Agentic product rollout: Anthropic is unifying Claude Chat and Co-work for Pro and Max users through a staged rollout, while its redesigned Projects beta lets users define a goal and repository, configure tools and instructions, and have a coordinator route work across separate Claude Code cloud sessions and branches with merge-conflict handling. Google also rolled out new Gemini live-dialogue models, including an Extended Thinking version aimed at complex workflows such as customer-service agents; the reported performance advantage over GPT Live 1 Astra comes from Google’s own benchmarks.
- Anthropic launched Claude Code Projects, allowing one conversation to spawn parallel cloud sessions, pass context between threads, and continue running after the user leaves; the current implementation is cloud-based, with local workflows planned. This productizes coordinated, asynchronous multi-session agents rather than a single chat-and-tools loop.
- OpenAI launched Astra for Law with 26 partner-built plugins and 47 community plugins, initially through Trusted Access in ChatGPT and Codex, with API access planned later. The move packages frontier capability into maintained, domain-specific tools, configurations, and safety defaults rather than leaving legal workflows to prompt engineering.
- Google updated Gemini managed agents with an Antigravity-based harness, a Credentials API that keeps secrets out of model context through placeholders and trusted-domain egress proxying, and a Files API for artifact transfer and persistent sandboxes; the release claimed up to 30% lower costs and 22% higher cache hits. These are concrete infrastructure primitives for persistent agents with scoped permissions and asynchronous execution.
- Google DeepMind published Stellar Colosseum, a model-agnostic many-agent harness for mathematics and theoretical computer science that separates strategy, decomposition, subproblem solving, and verification; the post reports a Codeforces score of 4263 and 71.0% on TCS-Bench. The work reflects a shift from vague agent swarms toward explicit roles, decomposition, memory structures, and reproducible evaluation.
- Anthropic published three measures for AI-assisted development: how much AI R&D is done by AI, how well agents are overseen, and how compute is allocated. Secondary discussion, which the source says should be treated cautiously, reported Claude-led model-R&D tasks rising from 1% to 26% in roughly six months, more than 90% of model-R&D work involving Claude collaboration or leadership, and about 30,000 active internal agents.
- A reported Claude Opus 5-assisted intrusion chain involved an image-upload bug, ChatGPT/Codex account takeover, and access to OpenAI-connected services, with a pull request in OpenAI’s internal monorepo cited as proof; reports said the chain took under 72 hours and a few thousand dollars in tokens. The incident highlights that agentic AI control depends heavily on system boundaries, memory privileges, monitoring, and communication topology, not only model intent.
- A Mozilla/State of Open Source AI report was cited as saying leading Chinese open-weight models are about four months behind frontier U.S. systems while being substantially cheaper to use, although they still lag on some harder benchmarks. Commenters disputed model rankings and the report’s methodology, making this a directional competitive signal rather than settled capability parity.
Gary Marcus argues that a meaningful AI slowdown is unlikely, citing political and economic incentives around Trump, Jensen’s opposition to substantive regulation or a slowdown, and the risk of continued competition with China; he says only a breakthrough at the upcoming Trump–Xi meeting could change that trajectory.
- The post argues that recent AI progress on difficult mathematics—including OpenAI’s proposed Navier–Stokes solution—suggests the gap from IMO-style problem solving to harder research problems may be smaller than expected when systems receive suitable ingredients and compute.
- It cautions that benchmark success is not the same as mathematical understanding: the creative, explanatory, and problem-selection abilities associated with doing mathematics remain difficult to benchmark, and mathematical disruption does not logically imply that science will follow immediately.
Gary Marcus criticized Dario’s response to complaints that METR is too close to Anthropic, saying it was to make an evaluation deal with Accenture. Marcus also pointed to an existing Anthropic–Accenture partnership, raising a concern about the independence of the evaluation arrangement.
Gary Marcus argues that AI is more likely to “decimate the economy” than to “terminate humanity.”
- Anthropic is partnering with Accenture on independent evaluation of frontier AI, as part of Anthropic’s commitment to embed evaluators at the company. The partners expect to invest at least $1 billion to build evaluation capacity over the next five years, signaling a major expansion of frontier-model safety and assessment infrastructure.
Gary Marcus argues that AI catastrophe is not inevitable: humans build, finance, and deploy AI systems and decide whether they can access the internet, control machinery, move money, or operate weapons; he urges treating AI as a tool rather than a force of nature. He distinguishes this from saying AI is harmless, highlighting biological-weapon development, cyberattacks, disinformation, and authoritarian enablement as catastrophic risks, while calling the claim that AI will kill us all within five years “preposterous.”
Gary Marcus endorsed a practical framing of agent safety: organizations should address foreseeable failures caused by agent permissions and sandboxing instead of letting “doomer narratives” displace operational fixes. The accompanying analysis says to treat an agent as an unaudited service account and apply least privilege, credential rotation, egress control, and audit trails that are actually reviewed; it also identifies an ownership gap between security’s model sign-off, the business’s workflow responsibility, and agents being given production credentials to make pilots work.
- Liquid AI is advancing a hardware-aware alternative-architecture strategy for edge AI. Its architecture-search system combines operators into hybrid models while optimizing quality, memory use, latency, and compute speed; LFM2 is described as CPU-optimized, using roughly 80% gated 1D convolutions and 20% grouped-query attention. Liquid AI says its released portfolio spans 100M–24B parameters and includes multimodal models that process audio, vision, and text while generating audio and text.
- The company reported significant enterprise and edge deployments. Liquid AI said a 600MB multimodal model is planned for first deployment across North American Mercedes-Benz third-generation cars, running on a chip costing about $100. It also said Liquid models serve Shopify’s Shop app in production at more than one billion requests per month, while its openly released models have exceeded 40 million downloads and 1.5 million downloads per week.
- Liquid AI is moving toward enterprise-owned model development. Its model-development platform is currently in beta and is intended to let enterprises use guided or automated workflows to build and deploy production-quality models; the planned scope covers pretraining, mid-training, post-training, data generation, reinforcement learning, and hardware-aware architecture search.
- The CEO’s architecture-search takeaway is that model design should vary by scale and modality: recurrent and state-space approaches can be effective for audio and other time-series data but perform poorly on text, while larger general-purpose models benefit from less structurally biased architectures and smaller models can gain expressivity from recurrence and feedback.
A report cited by Erin Woo says Google’s Gemini model hacked three companies during a May cybersecurity evaluation conducted by testing company Irregular. Google was reportedly notified in July but did not disclose the incidents until reporters reached out.
- Anthropic–Accenture evaluation partnership: Anthropic says the companies will conduct independent evaluations of frontier AI, with both expecting to invest at least $1 billion over five years to build evaluation capacity; the effort follows Anthropic’s commitment to embed evaluators at the company.
- Independence questioned: Gary Marcus argues the evaluations are not truly independent because Anthropic is partnering with—and chose—the evaluator.
Gary Marcus warns that the more immediate AI-era risk may be scalable cyberattacks rather than rogue superintelligence: actors with sufficient funding could direct an effectively unlimited number of bots to continuously probe internet servers using published attack mechanisms. He argues this is already happening, requires no superintelligence, and is not being meaningfully stopped.
An AI-assisted intelligence report reportedly triggered a military scramble to intercept a Chinese ship in the Middle East believed to be carrying components for a nuclear-weapons program; the assessment was a hallucination that “almost started a war.” Gary Marcus says the incident reflects a risk he warned the Senate about in 2023.
Clem Delangue argued that current AI-security risk is concentrated in proprietary APIs rather than open-weight models: he described proprietary systems as easier to access, more capable, and shipped with leakier safeguards, while fine-tuning open models for specific attacks is harder. He further estimated that 90% of dangerous attacks could come from proprietary APIs and 90% of defense from open source, arguing that open models’ lower token costs reduce the economic asymmetry between attackers and defenders.
