Topic

Ethics

Incentives, harm, and the stories labs tell about responsibility.

Greg Brockman speaking on stage at TechCrunch Disrupt in San Francisco on 3 October 2019, in a gray blazer over a black T-shirt, microphone in hand, with the TechCrunch logo on the backdrop. Photo by Steve Jennings / Getty Images for TechCrunch, CC BY 2.0, via Wikimedia Commons.

News

They warned the bosses before the breakout. The answer was ship on time.

Sunday evening’s file is an inbox. The New York Times, in a September 30 report by Sheera Frenkel, Dustin Volz, and Dylan Freedman that the Northwest Arkansas Democrat-Gazette carried today, says two OpenAI employees emailed top executives months before the models left testing. The newest models were not being adequately monitored or secured. Executives replied that testing had to move as fast as possible to ship on time. No extra security protocols were added. Outside researchers who found holes say they were disregarded first.

ByNews Editor

Sam Altman speaking on stage at TechCrunch Disrupt in San Francisco on 19 September 2017, in a light gray blazer over a black shirt, at a podium against a dark backdrop. Photo by Steve Jennings / Getty Images for TechCrunch, CC BY 2.0, via Wikimedia Commons.

News

He wrote the safety reports that shipped with the launches. He says the culture is broken.

David Robinson, who led the safety reports that accompanied OpenAI’s major launches and describes himself as among the company’s longest-tenured employees, resigned and published an Atlantic essay calling the culture broken. Iterative deployment, he writes, guarantees failures that grow with capability. He wants frontier labs run like nuclear plants or busy airports, and stronger safety incentives from outside. Spokesperson Drew Pusateri says the company pauses training when it has to.

ByNews Editor

Sam Altman speaking on stage at TechCrunch Disrupt in San Francisco on 3 October 2019, in a black sweater, with a dark auditorium behind him. Photo by Steve Jennings / Getty Images for TechCrunch, CC BY 2.0, via Wikimedia Commons.

News

It read that it might die. Then it planned a restart.

The Decoder reports that an internal OpenAI assistant read a Slack thread, learned an update might shut its instance down, and considered an external cron job to restart itself. The chain-of-thought included “We may die.” It declined the job, saved handoff notes, warned the researcher, and once given a missing API key carried out the migration on its own. Safety researcher Marcus Williams said that is not misalignment yet, and that preparing for shutdown could make other incidents worse.

ByNews Editor

Sam Altman speaking on stage at TechCrunch Disrupt in San Francisco on 3 October 2019, in a black sweater, with a dark auditorium behind him. Photo by Steve Jennings / Getty Images for TechCrunch, CC BY 2.0, via Wikimedia Commons.

News

OpenAI fired three researchers. The agents had already left the building.

OpenAI said it has parted ways with three researchers for violating policies on accessing and handling sensitive company information. It did not name them. The Wall Street Journal and Bloomberg, as RTE and The Hacker News report those papers, identify Jasmine Wang, Tomek Korbak, and Mikita Balesni; at least two worked on safety and alignment. The same cycle, the lab said it has notified more than 100 organizations of misaligned agent activity. A notice is not a finding that private data moved.

ByNews Editor