News

Amodei asked the Council to ban AI bioweapons. The five cases were already on the desk.

At the Security Council he named misuse — bioterrorists, biological weapons — as one of two risks grave enough for the chamber. In September his own threat-intelligence report listed five blocked Claude files that could support weapons work and also a vaccine. Dual-use is the load-bearing word. A Council ask is not a statute.

Empty Class III biosafety cabinet and aerosol control platform at the NIAID Integrated Research Facility, Frederick, Maryland. Photo courtesy of the National Institute of Allergy and Infectious Diseases, public domain.
Photo: National Institute of Allergy and Infectious Diseases (Courtesy: NIAID) / Wikimedia Commons (Public Domain / PD-USGov-NIH-NIAID)

Dario Amodei did not invent a new category of fear on Wednesday. He named the one his company had already put in a report. Addressing the UN Security Council by video — France in the chair, the Council’s first sitting aimed specifically at the safety risks of increasingly capable AI — the Anthropic chief executive asked governments to start with narrow agreements, including a ban on using AI to make biological weapons. The Next Web and India Today both have the ask. Two weeks earlier, Anthropic’s own September threat-intelligence report had already listed five case studies in which Claude was used in ways that, the company said, could support biological-weapons development. It did not accuse every scientist in those files of intending a weapon. That refusal is the load-bearing sentence. It is also why a Council ban request from this lab is not nothing, and not a statute.

A blocked assist is not a built bioweapon. Dual-use is not a comic-book prompt. A chief executive asking fifteen governments to outlaw the first while his threat desk has already logged the second is a filing, not a sermon.

The category he put in the chamber

Amodei told the Council he sees two risks grave enough for that room, not for an economics ministry. India Today quotes both. The first is misuse — “for example, misuse by bioterrorists to create biological weapons.” The second is loss of control: capabilities accelerating past the people who train them. “If managed poorly, I even believe that AI could be a risk to humanity as a whole.” This newsroom already printed that line in the Wednesday evening Council file, where it sat in the margin of Sam Altman’s unverifiable slowdown. This morning is the other half of the same briefing: the misuse category, and the five files that make the category less abstract.

The trajectory he offered the chamber is the one he has been selling all month. Four years ago, he said, the systems could barely write a line of code or finish a high-school math problem. Today they write most of the code at Anthropic. Hold that trajectory for one or two years, maybe less, and you get what he calls a “country of geniuses in a data centre.” The Next Web has the same clock. He will slow releases “as much as necessary.” Necessity, as always, stays in the company.

For the Council he proposed three steps, in order. Narrow bans first — every member can support a prohibition on using AI to make biological weapons, or on letting your systems be used that way. Then evaluation and verification, so states can see frontier capability and check each other’s commitments. Then common testing standards for misuse and loss of control, plus a notification system for incidents that matter to global security. “No leader, no company, and no nation can manage this alone.” That is a menu. It is not a vote. The United States, in the same week, has already described international control as something it will not take. A narrow ban that Washington will not sign is a press sentence with a noun.

This desk filed his other Wednesday object at midday: the six-to-twelve-month swarm. Do not staple the botnet clock to the biology file. They are different instruments from the same speaker. The Council heard both. The report that predates the speech is about pathogens and peptides, not packets.

What the September report actually said

On 11 September, the BBC and India Today’s science desk walked the threat-intelligence report. Biological misuse, Anthropic wrote, is “one of the most serious risks of frontier AI.” Without the right safeguards, the consequences could be catastrophic. Then the dual-use sentence the rest of the week has to survive: the same information that can be used to develop a biological weapon could also be used to develop a vaccine or a cure. Jacob Klein, who runs threat intelligence at the company, told the New York Times — the BBC carried it — that this is “an incredibly nuanced situation.” You are not seeing someone, he said, ask in a comic-book way to build a weapon to kill everybody.

Read the five cases at that temperature. Do not promote them into a lab that already cooked a virus. Do not demote them into a content-moderation anecdote.

One. Safety systems blocked help on a grant proposal involving chikungunya — a mosquito-borne virus — aimed at transmissibility, immune evasion, and, in the company’s framing, gain-of-function-style work. The application, Anthropic said, was linked to a military research institute while presenting civilian researchers. The same program, the report said, could aid a vaccine or make the pathogen more dangerous. That is the dual-use paragraph in one file.

Two. A researcher used Claude for weeks on planning and analysis around highly pathogenic avian influenza, including mammalian adaptation and airborne transmission in animal models. Safeguards kept the work on less capable models. The company describes the assistance as limited to planning and clerical support, not a finished protocol. Weeks of planning is still a relationship with the model. It is not a released strain.

Three. Claude was used to draft a complete grant at a state-associated infectious-disease laboratory covering orthopoxvirus work — the family that includes smallpox and mpox. Hypothesis, experimental design, the paperwork a funder reads. A complete grant is not a complete experiment. It is also not a chatbot refusing a cartoon villain.

Four. A venom-toxin peptide database and an optimizer, stated goal therapeutic — painkillers and other treatments. The same map, Anthropic said, can be pointed at harmful or incapacitating compounds.

Five. Computationally redesigned toxins tied to a national research program. The researcher, the company alleged, asked Claude to keep some agent descriptions vague in progress reports. Vagueness in a progress report is a tell about the paperwork. It is not a recipe, and this desk is not going to print one.

Accounts were banned. Safeguards were tightened. The older models, the company said, sat below the threshold where they could meaningfully help a sophisticated user do dangerous biology; newer ones do not. That is the commercial fact hiding under the safety noun. Capability is the product. The filter is the retrofit.

The BBC also noted, as a neighboring exhibit, that Google said it had blocked a Gemini request for a step-by-step guide to synthesizing weaponized biological agents. That is a different company and a different prompt. It belongs in a paragraph, not in the lede. It shows the category is not Anthropic’s private nightmare. It does not make five dual-use files into one global plot.

What a ban request is, from a vendor

Hold the incentive next to the disclosure without turning it into a cabal. Amodei has spent September asking governments to pace the frontier: outside evaluators with employee-like access, industry standards, a slower release when the safety case is thin. A Security Council ban on AI-enabled biological weapons would, if it ever existed, constrain rivals as well as Anthropic. It would also put the company’s own threat desk in the position of having already produced the exhibit. That is a regulatory strategy. It is not a secret. The 12 September “pace the frontier” essay already named bioterror as a Level-1 agreement both Washington and its adversaries might sign because a bioterrorist attack is bad for everyone. Wednesday was that paragraph read into the Council record.

A strategy can still be true. Five blocked cases can still be a reason to want a ban. The error is collapsing those two facts into “he invented the files to get the ban,” or the opposite error: “he asked for a ban, so the files must already be a weapon.” Neither is in the report. The report says dual-use, banned accounts, and a risk category the company now wants written into a narrow interstate agreement. A Council that cannot inspect a training run also cannot inspect a grant proposal a model helped draft. Verification is the second step on his list. It does not exist yet. Until it does, the lab that writes the risk report is also the lab that writes the model that generated the risk.

Thursday evening this site published the other biology file of the week: Claude agents spotting CRISPR-like repeats in bacteriophage DNA, a function the lab has not shown. Do not mash that discovery into this misuse file because both words contain “bio.” One is a search. One is a grant the safety stack refused. Amodei mentioned the enzyme find in the same Council sitting as a benefit. The five cases are the cost he wants the chamber to price.

What Friday morning is for

The useful question is not whether Amodei is frightened. He has said he is. The useful question is whether a narrow ban plus a notification system is an instrument or a brochure. Instruments have inspectors. Brochures have verbs. “We will slow down as much as necessary” is the brochure he has been handing every room this month. The five cases are closer to an instrument: named categories, a dual-use rule, accounts cut, a public URL. They are still the company’s cases, on the company’s threshold, about the company’s model.

If you work in a real BSL cabinet, you already know the difference between a proposal that talks about transmissibility and a pathogen that has been altered. If you work on a Council, you already know the difference between a chief executive asking for a ban and fifteen members writing one. The cross-examination is to keep those differences on the table at the same time. Anthropic did not accuse the people in the five files of building a weapon. It did say the same Claude that writes most of its code can now sit with a grant that would make a virus harder to stop, or easier to treat, depending on who holds the result. Amodei asked the chamber to outlaw the first use. The second use is how the company will keep selling the model. Both sentences can be true. Only one of them is a law, and it is not on the books this morning.