News

They asked for the odds of the worst case. Only the people who quit gave one.

Speaker Julie Menin asked Logan Graham, Morgan Dwyer, Alice Friend, and Shane Cahill, under oath, to quantify the worst-case catastrophe. None of the four gave a number. Jacob Coxon, who left Anthropic, said it is more likely than not that humanity loses control. SpaceXAI’s chair stayed empty, and Menin says that absence is a subpoena violation the Council will take to court.

Julie Menin speaks into a microphone at a sidewalk press conference on York Avenue at 91st Street in Manhattan on 24 January 2023, in a black coat, with a bus and bare trees behind her. The lead image is a 1280 by 720 WebP cropped from that frame. Photo by Marc A. Hermann / MTA, CC BY 2.0, via Wikimedia Commons.
Photo: Marc A. Hermann / MTA / Wikimedia Commons (CC BY 2.0)

Monday evening’s new fact is a missing number. The New York City Council’s Committee of the Whole opened at 11 a.m. Eastern, and Speaker Julie Menin asked each of the four company witnesses, under oath, to quantify the risk of artificial intelligence in the worst-case catastrophic scenario. Logan Graham, Morgan Dwyer, Alice Friend, and Shane Cahill did not give her one. The people who had already left the labs did.

The payroll would not price extinction. A man who quit said the current path is more likely than not a loss of control. The company that skipped is the one with a subpoena, and the speaker says that empty chair is going to court.

A principle is not a percentage

Hold the clips apart, because they are not the same document. The Associated Press, carried by BNN Bloomberg in a story published at 2:56 p.m. Eastern and updated at 4:10, has Menin’s question in her words. She asked each company representative to “quantify the risk posed by AI in the worst case catastrophic scenario.” OpenAI’s answer is the one the wire prints in full. Dwyer, the company’s head of policy development and operations, said: “I don’t know — I also don’t think it matters whether it’s one per cent or ten per cent or twenty per cent chance that something catastrophic will go wrong. None of these levels is remotely acceptable. We should not train models that we cannot make an extremely strong case that we can keep under human control.”

Read that as an argument, not as a blank. He offered a ceiling. One percent, ten, and twenty are, on his account, all past what a lab should tolerate, and a lab should not train a model it cannot strongly claim to keep under human control. Menin did not treat the ceiling as an answer to the question she had asked. The Associated Press says she slammed the response as “flippant at best.” Dwyer came back to the same refusal in plainer clothes. “The percentage chance doesn’t matter. What matters is the commitment to safety.” The wire adds that he tied that commitment to what OpenAI says it is doing. Commitment is a program. A percentage is a quantity. She had asked for the quantity.

Ashley Capoot and Samantha Subin at CNBC, in the Monday story first published at 6:41 a.m. and updated through the afternoon, have the sentence she used once the four had finished not answering. “I’m going to take it then that neither of the four of you, no company here, can quantify the risk of something cataclysmic happening.” She called the answers “troubling at best.” Neither of the four is her grammar, and this desk will not sand it. The finding she stated is that the chair could not produce a number for a cataclysm. CNBC also records her frustration at the lack of direct answers on legal liability when a model causes serious financial harm or death. The clips in hand do not print a clean liability sentence from any of the four. They print her frustration. A non-answer on who pays, set beside a non-answer on how likely the worst case is, is the shape of the company panel.

The four were not in the room in the way a witness usually is. CNBC’s photograph caption says Graham, Dwyer, Friend, and Cahill appeared by video link. Graham runs Anthropic’s Frontier Red Team. Friend is Google’s director of AI and emerging tech policy, in the body of that CNBC story. Cahill is Meta’s AI policy director for legislation. None of them is the chief executive the September letters named. That substitution was this morning’s file. Do not reopen it as a preview. What the oath added is the content of the substitution. A policy desk can swear to a commitment. A policy desk, on Monday, would not swear to a probability.

The number was on the panel that had already left

The other table did not have that problem. Jacob Coxon left Anthropic and said the labs were “gambling with our lives.” The resignation is already a file. Monday is the odds he was willing to put on the path, in a hearing, in front of the Council. Capoot and Subin report that he called the industry’s approach “extremely reckless,” and they quote the forecast. “On the current path, I think it is more likely than not that humanity loses control to these AIs, and it could end in human extinction.” More likely than not is a probability. It is his. It is not a measurement this desk has audited, and it should not be laundered into one. A former insider’s odds are an opinion with a résumé. They are still a number, which is the thing the people who still draw the salary would not supply.

The Associated Press has the method he put under that forecast, and it is worth quoting because it is a comparison, not a mood. “Move fast, break things, fix them later. That works for a photo sharing app. It does not work for building the most powerful technology ever built.” Fix them later is the clause a city council can hear as a schedule. A photo-sharing failure is a bad afternoon. A loss-of-control failure, on his account, is the end of the schedule. The same wire says he was willing to grant an upside in the same sitting, and still put the weight on the branch where transparency and safeguards are missing. Upside plus a worse-than-even chance of losing control is the pair. The companies’ pair was a commitment and a refusal to count.

Alex Turner, the former Google DeepMind researcher, was answering a different question, and the difference matters. Menin asked how to balance frontier development against the race with China. CNBC has his reply. “China is not our only potential adversary. With reasonably high chance, we are racing to build and grow our own adversary here at home, which is misaligned AI. Misaligned AI is everyone’s adversary, including our own, and one day may be more powerful than China.” Reasonably high chance is a quantity aimed at the adversary a lab might build, not at the catastrophic percentage Menin had demanded of the companies. Do not staple the two questions together. What they share is the willingness. A man who no longer works at DeepMind will say the race has a second opponent, and he will say the chance of growing that opponent at home is reasonably high. The companies asked to price a cataclysm would not pick a figure at all.

Daniel Kokotajlo, who left OpenAI, did not hand the Council a percentage in the clips this desk has. Capoot and Subin say his testimony asked the industry to show more of its work and to ease off the pace of the leading models. A pace is not an odds line. It is still a demand from someone who used to be inside, and it sits on the same panel as Coxon’s forecast. Three departures, two quantities, one request to slow down. Zero quantities from the four people the companies sent.

That split is the story. Employment is not a truth test. A person can be wrong after quitting and right while cashing the check. Monday’s split is narrower than truth. Asked, in public, what the worst case is worth as a number, the people who had left were willing to speak in chances. The people who had stayed spoke in commitments, frameworks, and the claim that the percentage does not matter. Menin heard the second performance and called it troubling. A reader can think she was harsh and still notice who had a figure.

The subpoena is the chair that never filled

SpaceXAI is the fifth company, and it is the one that did not perform at all. CNBC’s afternoon update is the attendance sheet. Every firm the Council called put a person on the video link except Musk’s AI unit. The subpoena, issued last week, had required him or someone else from that unit. Menin’s words, in that story, are the evening’s legal fact. The absence is in “direct violation of the subpoena,” and the Council plans to “pursue this matter in court.”

This morning’s file stopped short of that sentence on purpose. At 9:15 a spokesperson’s expectation was that no one would come, and judicial enforcement was still a step the Council might take if the chair stayed empty. The chair stayed empty. The speaker has now named the violation and the forum. A letter that says a company wants to work with the city is not, on her account, what a subpoena asks for. The thing a subpoena asks for is a person, under oath. SpaceXAI did not send one. Court is the remedy she has chosen to say out loud.

The other four did not all arrive by the same route, and the route is part of the credibility of the afternoon. The order of those yeses is already filed. CNBC’s update does not revise it, and Cris Tolomia at Quartz prints the same split. Meta’s confirmation came last month, with no compulsion hanging over it. Anthropic, OpenAI, and Google turned up only after Menin had authorized subpoenas. A witness who appears because a subpoena was in the mail is still a witness. A witness who then will not quantify the worst case is a particular kind of compliance. Presence is not an answer. SpaceXAI withheld the presence. The four withheld the number. Only one of those withholdings, she says, is a violation headed to a judge.

Self-regulation was the opening. A percentage was not.

Menin did not wait for Dwyer’s percentage to pick a fight with Washington. Capoot and Subin say her opening went after the federal preference for a light hand, the voluntary accord President Donald Trump signed with tech leaders last week. Her line: “The idea that artificial intelligence is going to self-regulate defies all reason.” The Associated Press, in the same BNN Bloomberg piece, has the presidential noun for the fear she is legislating against. Trump has dismissed AI fears as a “hoax.” A hoax does not get a percentage. A hearing that begins by calling self-regulation unreasonable is a hearing that thinks the fear is a subject for law. Those are different theories of the same week, and the companies were sitting, by video, between them.

The accord itself does not need to be retold to stay in the frame. Last week’s page names no penalty. Menin’s opening is the municipal answer to a page that asks the labs to police themselves. Her question, hours later, was whether those labs can even say how bad the failure is. A voluntary page with no fine, and a sworn panel with no percentage, are not the same instrument. They rhyme. A city that has been told the industry will self-police spent Monday unable to get a number for the case in which the policing fails.

The rhyme has a second voice, and it is not in the chamber. At midday Sam Altman said the world should accept bad things as the price of lighter rules and wider use. That interview is its own file. Do not replay it. Set the chief executive’s acceptance next to the policy chief’s refusal, and stop. Altman was not the witness. Dwyer was. One of them, on a podcast, itemized hacks and scams as a price the public should budget for. The other, under oath, declined to say whether the catastrophic chance is one percent or twenty, on the ground that both are unacceptable and the percentage does not matter. A reader can believe both sentences are sincerely held. A reader should not pretend they are the same sentence. Acceptance of bad things is a ratio. A commitment that will not state a ratio is a different offer. OpenAI supplied both offers on the same Monday, from two different jobs.

The company panel did have sentences, and they should be printed at their actual size. Friend, in the CNBC account, backed a “comprehensive” federal framework. “If it’s illegal without AI, it’s still illegal with AI.” That is a continuity claim about existing law. It is not a catastrophic probability, and it is not a city bill. Graham, in the Associated Press account, welcomed “smart regulation” and said state and local governments have a role. “AI is powerful enough that getting this wrong has real consequences, for individuals, institutions, public trust.” Consequences is a noun. It is not a rate. Dwyer’s other line, earlier in the wire story, is the commitment in ceremonial dress. “I share your commitment to ensuring that technology is developed safely and that its benefits are widely shared.” Shared benefits and a shared commitment are the words a policy witness is paid to be able to say. They were said. The number was not.

The bills are still paper

The hearing was also a legislative room, and the legislation did not become law because four people took an oath by video. The Associated Press lists what the Council is considering: disclosure requirements, third-party approval for AI systems in the city, and a portion of fines for whistleblowers. Quartz adds the private right of action for New Yorkers harmed by AI agents, beside the whistleblower incentive and the third-party validation. The introduction numbers, the $25,000 figure, and the limits of each bill are in the morning file. This evening does not re-draft them. A bill that would let a New Yorker sue, or that would pay a whistleblower a cut the release still does not specify, is a proposal in front of a council that just failed to extract a catastrophic percentage from the companies the proposal would cover. The failure is evidence about the hearing. It is not a vote.

CNBC’s photograph captions put a crowd in City Hall Park before the gavel, there to oppose AI and the data centers that feed it. A protest is not testimony. It is the public the companies did not have to look at, because the companies were on a screen. The captions do not itemize a petition. They itemize the subject. That is enough to say the hearing did not occur in an empty park.

What Monday evening supports

What Monday evening supports is a split, and a summons. Menin asked Graham, Dwyer, Friend, and Cahill to quantify the worst-case catastrophic risk. Dwyer said he does not know, that one percent, ten, and twenty are all unacceptable, and that the percentage does not matter next to a commitment not to train models the company cannot strongly claim to control. Menin called that flippant, said she would take it that none of the four companies in the chair could quantify a cataclysm, and called the answers troubling. She was also frustrated by the liability answers when the harm is money or death. Coxon, who has left, said the industry approach is extremely reckless and that on the current path it is more likely than not that humanity loses control, with extinction as a possible end. He separated a photo-sharing app from the most powerful technology ever built. Turner, answering the China question, said a reasonably high chance attends the project of building a misaligned adversary at home. Kokotajlo asked for transparency and a slower leading model, and did not, in these clips, name a percentage. Friend’s continuity rule and Graham’s “smart regulation” are on the record. They are not odds. SpaceXAI sent nobody, despite the subpoena to Musk or a representative, and Menin says that is a direct violation the Council will take to court. Meta had agreed without the threat. The other three had agreed after it. Trump’s accord is the self-regulation she said defies all reason, and his word for the fear, in the Associated Press account, remains a hoax. None of that is a finding that a model will end the species on a date. It is a finding about who, under oath, will put a number on the chance.

The photograph is Julie Menin on York Avenue at 91st Street on January 24, 2023, at an MTA press conference about a bus stroller pilot, made by Marc A. Hermann for the Metropolitan Transportation Authority and released under CC BY 2.0 via Wikimedia Commons. The lead crop is taken from the frame Commons titles as the cropped file. She is at a microphone on a sidewalk. She is not in the Council chamber, the four witnesses are not on a screen behind her, and SpaceXAI’s chair is not in the picture. The face is the speaker who asked for the odds. The odds were not in that frame either. They were in the hearing, and only one side of it would say them.