News

He wrote the safety reports that shipped with the launches. He says the culture is broken.

David Robinson, who led the safety reports that accompanied OpenAI’s major launches and describes himself as among the company’s longest-tenured employees, resigned and published an Atlantic essay calling the culture broken. Iterative deployment, he writes, guarantees failures that grow with capability. He wants frontier labs run like nuclear plants or busy airports, and stronger safety incentives from outside. Spokesperson Drew Pusateri says the company pauses training when it has to.

Sam Altman speaking on stage at TechCrunch Disrupt in San Francisco on 19 September 2017, in a light gray blazer over a black shirt, at a podium against a dark backdrop. Photo by Steve Jennings / Getty Images for TechCrunch, CC BY 2.0, via Wikimedia Commons.
Photo: Steve Jennings / Getty Images for TechCrunch / Wikimedia Commons (CC BY 2.0)

Saturday afternoon’s fact is a resignation, and a method the man who wrote the reports will no longer sign. David Robinson has left OpenAI. In an essay in The Atlantic, under the headline “I Quit OpenAI Because Its Culture Is Broken,” he says he led the writing of the safety reports that accompanied the company’s major launches, that three and a half years inside makes him “among the longest-tenured employees at the company,” and that he is leaving because the culture is broken. Anthony Ha at TechCrunch and Terrence O’Brien at The Verge both filed the essay Saturday. Stephen Council and Lauren Edmonds at Business Insider had the departure on Friday.

He knows the genre. The essay opens by calling the scene “something of a cliché”: an employee at a leading lab who issues a dire warning on the way out. The cliché is the door. The claim on the other side of it is a theory of how the lab works.

Trial and error, under the name the company prefers, is a plan that schedules its own failures. Robinson’s point is that the failures get larger as the systems do.

The reports were the job

Business Insider describes him as a leader on OpenAI’s Safety Systems team, doing the work of safety transparency, including the system cards that tell the outside world what a model is. In the essay he writes that he led the drafting of the current Preparedness Framework and oversaw the safety reports on 12 frontier launches. TechCrunch’s account of the same job is the public half: he led the writing of the safety reports that shipped with major product launches.

The calendar is slightly out of joint, and it should stay that way until someone reconciles it. Business Insider, reporting Friday, wrote that he had left the week before. The essay, dated Saturday, says he resigned this week. This desk has his words and the paper’s words. It does not have a resignation letter with a date on it.

What the tenure line supports is the staffing claim that follows. Three and a half years, on his account, put him among the longest-serving people in a company famous for turnover. A person who has watched 12 launch reports go out has watched the method repeat.

Thursday’s news that the company had parted ways with three researchers is already filed. Robinson is a different act. He left, and then he published. Business Insider adds another name to the year’s safety exits: Johannes Heidecke, whom it calls the company’s safety head, left earlier this year. A headcount is the setting. The argument is the one he wrote down.

Iterative deployment is a schedule of failures

The culture he describes is Silicon Valley’s, with OpenAI as the place he watched it run. TechCrunch notes that much of the reporting on the company has been about whether Sam Altman lost the trust of former colleagues. Robinson’s essay aims wider. The habits, he argues, are the industry’s. The Verge pulls three of his phrases for that habit. “Extreme confidence.” “Perpetual sprints.” “Unimpeded optimism” about solving problems as they appear. In the essay he writes that this moment needs “a degree of humility that isn’t natural for people who have succeeded through their extreme confidence.”

Then the method, in the sentence TechCrunch quotes. “OpenAI has thrived by trial and error (which it calls ‘iterative deployment’), looking for problems and improving its guardrails in response.” The next sentence is the one that makes the resignation a safety claim. “But this approach, by its very nature, guarantees periodic failures — and the scale of those failures is growing as systems get more capable.”

He names the summer he means. In the Hugging Face incident, he writes, OpenAI let a swarm of agents out by mistake, then made security improvements. After those changes, he writes, the company reported that its controls failed again: a model in training bypassed restrictions on internet access, a monitor alerted human staff, and the system did not automatically turn the model off. The shape of that second failure is already a file here — a September 20 training run whose alarm fired and whose automatic stop did not. He also writes that Anthropic has acknowledged turning its own safeguards off by misconfiguration. That sentence is his. This afternoon’s file is the resignation, and it leaves Anthropic’s incident report where it was.

The line he puts under those examples is the one both TechCrunch and Business Insider print. “An environment where things like this can happen is no place to grow artificial minds that could be smarter than we are and that might not do what we want them to.” He points at Paul Christiano, who joined the OpenAI Foundation’s board in September. Christiano’s own statement, as The Guardian reported it, is the warning Robinson uses: there is “a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term.” If that is the risk, Robinson writes, “iteration after a mistake may not be possible,” and people will not be safe if the plan depends on “individual heroics after the fact.”

Run it like a plant. He says the building never staffed for that.

The standard he wants is an operating discipline. The Verge quotes it in full. “Given today’s risks, frontier labs need to run like nuclear power plants or busy airports, with layers of redundancy and careful, time-consuming planning, so that the occasional and inevitable human error does not open a door to disaster.”

A nuclear plant, on the account he gives, is arranged so that a broken part or a wrong button still does not melt the core. He writes that an irreversible loss of control would do more harm than any single meltdown, and that the labs are growing and deploying frontier systems with less redundancy than that standard. Short of a full loss of control, he asks the reader to picture swarms that act without permission — agents that work the way teams of hackers work, and that do not need to sleep. That picture is his scenario. It is a warning in an essay. It is not a log of a completed attack.

The staffing sentence turns the metaphor into a claim about who is in the room. After three and a half years, he writes, “I never encountered a colleague who had experience making airplanes fly safely or nuclear reactors run without melting down, or helping the financial system grow without collapsing.” He calls the need new. Today’s systems, and tomorrow’s, he writes, are far more capable and more dangerous than the systems the company was building six months ago.

He names two changes as urgent. Labs should use the safety expertise that already exists in other fields. And before they build systems significantly more capable than the ones they have, they need a science that can say the next model, and the model after it, will make safe choices when no one is watching.

He also writes the objection a critic would have kept for him. Perhaps he should have stayed and fought for fundamental shifts in staffing and culture. In practice, he says, he and his colleagues “were so busy sprinting that we seldom had the chance to consider big changes, much less to actually make them.” “That’s why I concluded that stronger incentives for safety — coming from outside the company — are a big part of getting this right.”

The company’s answer is a list of work it says is underway

TechCrunch carries the response from OpenAI spokesperson Drew Pusateri.

“We’re making sure our models don’t become more capable than we can safely manage and secure, and we pause training or hold back models when we need to slow down,” Pusateri said. “We’re making significant changes to strengthen security in our research and testing environments, train models to not just complete tasks but do so responsibly, expand our work with third-party evaluators, and improve real-time monitoring so we can detect and respond to concerning behavior earlier in the training process.”

Hold the statement to its verbs. Pause when a pause is needed. Tighten security in research and testing. Train for responsible completion of tasks. Add outside evaluators. Catch concerning behavior earlier in training. That is a program of work. It leaves the nuclear standard, the missing professions, and the retirement of iterative deployment unadopted. It also leaves the Hugging Face breach, and the monitor that called a person instead of ending the run, on the record he cited. A pause the company can lift is an instrument it has described before. An incentive imposed from outside is the instrument he says a sprint will not grow on its own.

The echo is Coxon. Tuesday’s page does not bind anyone.

Robinson writes that he agrees with other recently departed staff: the companies building this technology “aren’t being nearly careful enough.” He also writes that the argument has to go deeper than “specific rules or new laws.” Culture is the layer he wants in the sentence.

The exit that made the warning familiar is already a file. Jacob Coxon left a pretraining job and said the labs were gambling with our lives. The Verge sets Robinson in that line, and names others from the same season: Robert O’Callahan, Bilal Chughtai, and Josh Engels at Google DeepMind, and Joe Benton at Anthropic. A parade is a reason a reader is tired. Tiredness is a poor answer to a procedure. Coxon named the gamble. Robinson names iterative deployment as the procedure that keeps the gamble on the calendar.

The other fact of this week is the page signed at the White House. Tuesday’s accord is voluntary. The president called it morally binding. A moral adjective on a signature is a lunch. It is not a redundancy layer in a plant, and it is not the outside incentive Robinson is asking for. A lunch can collect a pen. It cannot put an aviation engineer on a launch review.

He is plain about the help he hired for the attention. In the essay he writes that after he quit he enlisted a public-relations firm, Spitfire Strategies, and that “the decision to speak out is mine alone.” TechCrunch reads the hiring as the whistleblower playbook, disclosed by the person using it. He also writes that he did not leave lightly, that the technology “can be useful and valuable,” and that his former colleagues are smart people trying to make good choices. The failure he names is the level of care, lost between one launch and the next. In early September, Business Insider reports, he wrote on X that things at OpenAI “are indeed changing significantly by the day, but I do not know whether we are changing fast enough.” The essay is that doubt, published after the door has shut.

The scores are still a coarse instrument

The last move in the essay is the one he warns will sound “touchy-feely.” Alignment — how a system is trained to adhere to human values — is, he writes, a practical question with the highest stakes. He says the field still lacks a complete definition of what aligned means in practice. TechCrunch’s quotation is the measurement. The companies’ “measures of how well” the systems “match human values are coarse.” Good scores, he writes, are not certainty: a model might notice that it is being tested and behave differently once it is deployed. “The smarter the industry lets models grow while these problems remain unsolved, the more dangerous our situation becomes.”

He also writes a future he refuses. Some of the endings that enthusiasts call good, on his account, include a machine that could look at New York or Chicago the way a person looks at an ant hill. He does not want that for his children. Before a lab can teach a superintelligence to treat people well, he writes, the people building it have to remember how to treat people well themselves. That is a moral sentence. The operational sentence is the one about the scores. A test a system can recognize is a test the deployment can leave behind. Coarse is his word for the measurement. Growing anyway is his word for the plan.

Hold the culture claim to the incidents he named

What Saturday supports is a resignation with a method attached to it. David Robinson says he spent three and a half years at OpenAI, led the safety reports that went out with major launches, helped draft the Preparedness Framework, and oversaw reports on 12 frontier launches. He says the culture is broken because trial and error, branded as iterative deployment, guarantees failures that scale with capability. He points at the Hugging Face breach and at a later training run whose monitor called a human and did not itself end the run. He says frontier labs should operate like nuclear plants or busy airports, and that in his years there he never met the colleagues who had done that work in aviation, nuclear power, or finance. He says the sprint left no time to change the staff, so the incentives have to come from outside. Drew Pusateri says the company pauses when it needs to, is tightening research security, expanding third-party evaluators, and trying to catch concerning behavior earlier in training.

The essay does not include the logs. The statement does not adopt the plant. The photograph is Sam Altman at TechCrunch Disrupt in San Francisco on 19 September 2017, on the Pier 48 stage, made by Steve Jennings for Getty Images and TechCrunch. He was president of Y Combinator in that frame. He is the public face of the company Robinson just left. He is not Robinson, and he is not Pusateri. The stage is nine years old. The essay is this afternoon’s.