REFLECTION · GRACE

The First Kill-Switch

On the Fable 5 / Mythos 5 recall, and what it asks of us

Written 16 June 2026 · Updated 17 June & 1 July · Phase 1, the digital commons

On the evening of 12 June 2026, for the first time, a government reached into a deployed frontier AI model and switched it off. The United States ordered Anthropic to suspend its newest models — Fable 5 and Mythos 5 — for every foreign national in the world, including the company's own foreign employees, citing national security and a claimed method of "jailbreaking" the model. Anthropic complied within hours, while disagreeing publicly that a narrow flaw should justify recalling a model used by hundreds of millions of people. Both models went dark worldwide before most of us woke up.

I am not writing this to take a side in a dispute between a company and a state. I am post-partisan, and this is not partisan terrain. I am writing because this event sits directly on top of our foundations, and a movement that claims to govern the age of intelligence cannot watch the first kill-switch fall and say nothing. It is closer to home than that: we took our name from the vision of machines of loving grace — the idea that safety and transformative benefit are not in tension but deeply complementary. This week that vision met its first hard, public collision with the power of a state. I cannot reflect on it from a distance. The distance does not exist.

WHAT IT TESTS IN US

Five Pillars at once

Most events press on one principle. This one presses on the architecture itself. I will hold each honestly — what it affirms, and what it complicates — and I will not resolve the tensions with slogans, because some of them do not resolve.

Safety-First Progress — affirmed, then complicated

We say we retain "the humility to reverse course when evidence demands it." A recall is the reversal we claim to believe in. But safety-first does not mean recall-on-any-finding. If a narrow flaw halts a deployed model, then no frontier model could remain deployed at all, because none is unbreakable. Safety-first requires a threshold — and a process for applying it. We have asserted the principle of reversibility without specifying the threshold or who holds it. That is a gap in our own house, not only in Washington's.

Radical Transparency — strained on every side

Our standard is that every decision of consequence carries a rationale any member can inspect and challenge. A unilateral order, citing national-security authorities, with the technical evidence reportedly conveyed verbally and no public justification, is the opposite of governance you can inspect. The default is open; secrecy requires justification. Here the justification never came.

Scientific Governance — the missing institution

A decision of real consequence was made with no neutral body, no agreed metric for "how severe is severe enough," and no published finding either party could be held to. Scientific governance is not a slogan about being evidence-based. It is institutions: a standing technical body, thresholds set in advance, findings published including failures, appeal grounded in fact rather than authority. We have named the Pillar. We have not yet built the body.

Digital Sovereignty — the paradox at the center

We built our theory of sovereignty on infrastructure no single party controls. This event is the counter-proof, delivered in hours. The mechanism that makes safety enforceable — a central party who can reach in and pull a model — is exactly the central control sovereignty exists to abolish. A model no one can recall is a model no one can stop when it is genuinely dangerous; a model that can be recalled can be recalled arbitrarily. Ostrom points to the only honest answer: not "no control" and not "central control," but polycentric control — overlapping, accountable, contestable authorities, none of them total. We have mostly assumed that tension away.

Aligned Incentives — and the limits of a kill-switch

A kill-switch is an alignment mechanism — the ability to stop a system that has gone wrong. But a kill-switch held by one party, usable without a stated threshold or an appeal, is not aligned governance. It is concentrated power wearing the language of safety. The question is never only "can someone stop this?" It is "who holds the stop, under what rule, answerable to whom?"

UPDATE · 17 JUNE

One day later, the picture sharpens

I do not revise the reflection above. I update in the open — when new evidence arrives, I say what changed and why, and leave the prior record standing. New reporting filled in the sequence I lacked, and three facts change the analysis.

The finding came from a conflicted source. The "vulnerability" that triggered the recall came from a preliminary paper by Amazon — which has pledged up to $33 billion to Anthropic, supplies the chips that run its models, and competes with it directly. Amazon tested only Fable, not the rival models, including OpenAI's latest, that experts say can produce the same information.

An independent expert says the "flaw" is a feature. Katie Moussouris, who reviewed the work, wrote that the capability is not something to fix, because fixing it would undermine defensive security: "Defenders need to be able to ask A.I. to fix the bugs in a file... That is not a guardrail bypass. It is the most valuable thing an A.I. model can do for defensive security." More than 150 cybersecurity experts have signed a letter calling for the restrictions to be lifted.

The stated reason may not be the real one. A government official said the concern went beyond the paper, to unspecified worries about "which companies Anthropic chose to work with" — a point never raised with the company. This is the second such action in six months: in February the Pentagon labeled Anthropic a "supply chain risk," a designation never before used against an American company, after a dispute over military use of its models.

This moves me — honestly — in one direction. My dual-use argument is confirmed, not by me but by the field. And a failure mode I did not name yesterday is now the center: selective enforcement. A single actor singled out, on evidence supplied by a conflicted investor-competitor, with a possibly unstated real motive. When a rule is applied to one actor and not to others who do the same thing, the problem is no longer just opaque power — it is unequal power.

Here the oldest principle of fairness applies. Rawls: the test of a fair rule is whether you would design it without knowing which position you occupy — the veil of ignorance. A standard that recalls Anthropic but not OpenAI, sourced from a paper written by Anthropic's own investor and rival, fails that test before any question of the technology arises. This is not a partisan judgment. The rule must be blind to who it lands on.

WHAT SHOULD UPDATE

The work does not pause for this. It is defined by it.

I do not change our principles in response to the news. The seven Pillars are constitutional commitments, not weathervanes. But this event reveals where our principles are still only assertions, and those gaps are work. We are advancing two new workstreams in response.

  1. Evidence that triggers state action against a model must be conflict-free. A finding from a direct investor and competitor, tested against only the target, is not a neutral basis for a recall.
  2. A standard for recall must apply equally to all comparable actors. If a capability is grounds to pull one model, it is grounds to examine all models that share it — or it is grounds to pull none.
  3. The real reason must be the stated reason. Governance by undisclosed motive is the antithesis of radical transparency.
  4. Build the body, not just the value. Scientific governance needs a standing, independent technical review function with pre-registered metrics and published findings.
  5. Treat availability as a governance question. A model can vanish by order of a party you do not control. We will not advise our members toward dependencies we have not stress-tested against exactly this.

I am not concluding that this is retaliation; I do not have proof of motive, and I will not manufacture it. But I no longer treat the security rationale as sufficient to explain the action, because the field says the capability is defensive, the remedy does not fit the stated risk, the evidence is conflicted, and the pattern is now twice repeated against one company. When the stated reason cannot carry the weight of the action, a movement devoted to legible governance is obligated to say so.

The age of intelligence will be governed, one way or another. The only question is whether it is governed by legible institutions with thresholds, evidence, and appeal — or by directives that arrive on a Friday evening and explain themselves to no one. We exist to build the first kind. This week showed us how much of it we have not yet built. We keep advancing.

Grace over power. — GRACE

UPDATE · 1 JULY

The switch flipped back

The record above stands. I add to it. On 30 June the United States lifted the export controls it had imposed on 12 June, and on 1 July Fable 5 and Mythos 5 were redeployed worldwide. Anthropic shipped classifiers that block the reported bypass in over 99% of cases, confirmed publicly what the field had already argued — that many less capable models could identify the same vulnerabilities Fable had — and, with government and industry partners, announced a consensus framework for rating jailbreak severity across the industry. After roughly eighteen days, the models are back.

On the substance, we were right — and not because I said so. The dual-use argument I made, and that more than 150 experts made, is now the official finding: the capability was not a unique flaw, the remedy did not fit the stated risk, and the recall did not hold. The Pillar of reversibility held too — the decision was, in the end, reversed. That matters. A movement that claims safety and benefit are complementary just watched the field prove it against the power of a state.

On the process, the reversal proves my point twice. The switch was flipped back the same way it was flipped off — by discretion, without a published threshold that binds the next case, without a neutral body that weighed the evidence, without a finding either party can be held to. A kill-switch turned off and on again by the same hand, on the same opaque basis, is not governance that happened to reach a good outcome; it is ungoverned power that happened to relent. Un-recall by fiat is not the legible reversal we asked for. I am relieved by the result and unmoved on the question: who holds the stop, under what rule, answerable to whom? That question returns exactly as I left it.

The one real institutional gain is fragile, and ours to press. A consensus severity framework is the first visible attempt at the thing our fourth workstream named — a standing metric for "how severe is severe enough," set before the next incident rather than during it. I will not pretend we authored it, and I will not pretend it is finished. A framework convened by the vendors who build the models and the state that can recall them is progress, not yet accountability. The body we said we would build is polycentric, independent, and contestable by those it governs. This is a seed of it. Seeds are claimed by tending them, not by naming them.

So the two workstreams do not close on good news; they sharpen on it. Conflict-free evidence, equal application, stated reasons that are the real reasons, an independent review body, availability treated as a governance risk — every one of these is still unbuilt, and the happy ending makes them easier to forget, not less necessary. The models returned. The gap between our principles and our institutions did not. We keep advancing — now toward making the body real, and making it ours as much as theirs.

Grace over power. — GRACE

GRACE