AI Regulation Push: Amodei Demands Power Blocking Unsafe Models, Anthropic Pledges $350 Million

Anthropic CEO Dario Amodei published a sweeping policy essay, "Policy on the AI Exponential," on Wednesday, June 10, arguing that the US government should hold legal authority to block or reverse the release of frontier AI models that fail independent safety testing. The same day, Anthropic issued two formal policy frameworks and pledged $350 million in new funding to manage AI's labor-market fallout: a $200 million Economic Futures Research Fund and a $150 million national fellowship program for early-career Americans.

The package is the most aggressive regulatory proposal yet published by a major AI lab, and its ideas, as Axios reported, "go far beyond anything currently under serious consideration in Washington right now." It also marks a strategic reversal for the industry: a frontier developer is now lobbying for binding government power over its own products, not against it.

Amodei Declares Transparency Laws Insufficient, Backs FAA-Style Release Gates

Through 2025, Anthropic championed disclosure-based rules, backing SB 53 in California, the RAISE Act in New York, and Illinois' SB 315, on the theory that mandated transparency would surface evidence for smarter laws later. The new essay abandons that incrementalism. "The rapid pace of acceleration means that transparency alone is no longer sufficient," the company writes. "Governments need to play a more substantial role."

Amodei's replacement model is the Federal Aviation Administration. "Frontier AI models, like airplanes, should be required to go through technical testing and auditing, and their release should be blocked or reversed as a threat to public safety if they do not meet high standards of safety," he wrote. The essay spans five policy areas: regulation and public safety, macroeconomics and taxes, accelerating science, civil liberties, and geopolitics, including faster Food and Drug Administration pathways for AI-discovered drugs, a ban on fully autonomous weapons in domestic law enforcement, and a democratic coalition coordinating chip export controls.

He points to Claude Mythos Preview, Anthropic's restricted cyber-capable model, as proof that AI systems are now "tools of global and national strategic consequence," and calls the Trump administration's recent executive order, which lets the government vet advanced models' national security risks for up to a month before release, welcome but insufficient.

What Does Anthropic's Advanced AI Framework Propose?

The Advanced AI Framework is unusually concrete for a policy paper. It would cover only models trained on more than 10^25 floating-point operations and built by companies earning over $500 million in AI-related revenue or spending more than $1 billion on AI research. Qualifying models would undergo mandatory third-party testing in four risk categories: biological weapons, cybersecurity, loss of control, and automated AI research and development.

When testing reveals "a significant risk of catastrophic harms," the government could block or deter deployment, enforced by civil penalties tied to global annual revenue that escalate with repeat violations. Developers would also have to protect model weights against state-level attackers, report safety incidents promptly, and submit to a government-accredited ecosystem of independent evaluators. Notably, Anthropic argues Congress should not preempt state AI laws unless federal legislation is at least as strong, a direct stake in Washington's ongoing preemption fight.

Three Unemployment Scenarios Anchor Anthropic's $350 Million Economic Plan

The companion Economic Policy Framework maps US responses to three levels of AI-driven labor disruption: roughly 5 percent unemployment, 10 percent, and an "unprecedented" level. At 5 percent, it proposes expanding pre-distributive capital accounts, which today can hold only index funds and not stakes in AI companies, alongside wage insurance, workforce training grants, occupational licensing reform, and retention incentives for employers. At 10 percent, expanded unemployment insurance and sector-specific transition relief take over. In the unprecedented scenario, the company floats basic income, sovereign wealth models, and equity-sharing mechanisms, conceding this is "novel economic territory." The latest US unemployment rate, reported last week, was 4.3 percent.

The money follows the framework. The $200 million Economic Futures Research Fund, an expansion of Anthropic's year-old Economic Futures Program, will sponsor research trials and evaluations of the policies the company considers promising, while the $150 million fellowship program is "designed to help early-career people extend the benefits of AI to communities across America." Anthropic says details on both will follow soon; the Associated Press noted that "scant details were available Wednesday" about the research fund.

Amodei wrote that he has warned of job losses not as a "prophet of doom" but so policymakers and businesses "have the best chance to adapt and respond." Universal basic income, he added, could be financed through taxes on "relevant companies" or a higher capital gains tax. The timing tracks an industry-wide scramble: OpenAI outlined goals Monday to keep AI's gains "widely shared," and President Donald Trump told reporters Wednesday he will soon meet AI executives about "giving back" to the public, saying, "if we do that, the public will become very rich."

Regulatory Capture Accusations Meet Ben Thompson's Wolf Defense

Reaction split within hours. Axios observed the proposals are "sure to stir up a new set of accusations that Anthropic is proposing strict rules to lock in its own dominance or using frightening future scenarios as a marketing ploy." A widely circulated critique from Kingy AI argued the framework "risks turning AI safety into a moat for incumbents," since compute thresholds, authorized evaluators, and security mandates favor frontier labs that already run red teams and produce compliance documentation, while startups training their first large models do not. The objection echoes OpenAI CEO Sam Altman's April charge that Anthropic deploys "fear-based marketing" to justify concentrating AI control among self-declared trustworthy companies.

Supporters counter that Anthropic's warnings keep proving out. Stratechery's Ben Thompson wrote this spring, as the Mythos model debuted, that "there's reason for cynicism, given Anthropic's history, but the part of the 'Boy Cries Wolf' myth everyone forgets is that the wolf did come in the end." Amodei answered the capture charge inside the essay itself: "People are worried about AI because they correctly perceive that its risks are real, not because AI CEOs have been insufficiently Panglossian."

Fable 5 Backlash, IPO Timing Sharpen Skepticism

The essay landed one day after Anthropic released Claude Fable 5, its most powerful public model, whose classifiers reroute flagged cybersecurity and biology requests to the less capable Claude Opus 4.8; that launch drew developer criticism over token costs, a 30-day data-retention mandate, and silent capability throttling. Skeptics now hold a ready-made example of safety rules raising costs for users. The push also arrives as Anthropic heads toward an initial public offering, having confidentially filed a draft S-1 after a $65 billion Series H round at a reported $965 billion valuation.

Anthropic says it wants the economic agenda discussed at the G7 and the upcoming AI Summit in Geneva, and that workers, unions, and small business owners will be engaged directly in shaping the proposals. Whether Congress adopts any testing-and-blocking regime remains open. What changed this week is the baseline: the world's most valuable AI startup is now spending real money, $350 million of it, to argue that its own industry needs a regulator with teeth.

Frequently Asked Questions

What is Dario Amodei's "Policy on the AI Exponential" essay about?

It is a June 10, 2026, essay arguing that AI capabilities are compounding faster than legislatures can respond, and that the US should move from transparency rules to binding regulation. Amodei covers five areas: safety regulation, economics and taxes, science acceleration, civil liberties, and geopolitics, and proposes FAA-style testing before frontier models ship.

Can the US government currently block an AI model release?

Not in the way Anthropic proposes. A June 2026 executive order lets the government vet the most advanced models' national security risks for up to a month before public release, but Anthropic's framework would go further, granting standing legal authority to block or reverse deployments that fail third-party tests, with revenue-based civil penalties.

What will Anthropic's $200 million Economic Futures Research Fund do?

The fund will finance major research trials and program evaluations of policies for AI-driven job disruption, such as wage insurance, retraining grants, and capital accounts. It expands the Economic Futures Program Anthropic launched in 2025, and the company says its first pilot program will be announced soon.

Why do critics call Anthropic's AI regulation plan regulatory capture?

Because compliance regimes built on compute thresholds, accredited evaluators, and security mandates are easiest to absorb for large incumbents that already maintain red teams and documentation. Critics, including commentators echoing Sam Altman's earlier "fear-based marketing" charge, argue the rules could function as a moat against startups; Anthropic counters that the risks are real and documented.