AI chiefs split over who should police the machines

Sam Altman believes the world should trust him. Speaking in San Francisco on Tuesday at an annual conference hosted by the software firm Salesforce, the chief executive of OpenAI told his audience that the public ought to have more faith in his company and its competitors to act responsibly as artificial intelligence grows more powerful. The world, he said, should trust that the industry will do the right thing, because it is the right thing to do and because those building the technology feel the weight of it.

The reassurance came wrapped in an admission. Asked about mounting public anxiety, Altman conceded that fear is rational. It does not take as much imagination as it once did, he said, to picture how this could go wrong. The world, he added, is right to be afraid.

His remarks were his first in public since a researcher who left the AI firm Anthropic published a viral post last week claiming that the technology could kill all humans by the end of the decade if left unchecked. Several executives and experts said they agreed, though none explained how they had reached that conclusion or by what mechanism such a catastrophe might unfold.

The intervention has sharpened a quarrel that now runs through the heart of the industry: whether the companies building the most consequential technology of the age can be trusted to restrain themselves. Mark Zuckerberg, the Meta chief executive, argued on X that every AI firm has both the ability and the incentive to act on safety, since any laboratory that neglects alignment will fall behind and any that causes harm faces significant liability. Alignment, the effort to build human ethical principles into AI systems, has become the industry's favoured term for keeping the machines on track.

Others went further. Jensen Huang, whose company Nvidia is the most valuable in the world, told the same conference that AI firms should decide for themselves when new versions of the technology are released, and that outside forces should not manage them. We do not need new laws or regulations, he said, rejecting what he called a false choice between the speed of innovation and the safety of AI products. Safety, in his account, is an engineering problem: companies should simply pause if they lose confidence in a product.

That position is not universally held. Dario Amodei, the chief executive of Anthropic, has called for the pace of development to slow and for governments to regulate the industry, and his firm is now in what he described as a dialogue with the rest of the industry about committing to better safety standards. One of the surprises of the AI boom, Amodei said during a cameo at the conference, was not how fast the technology advanced but how quickly the companies behind it became central to everything.

Political opinion is no more settled. Senator Bernie Sanders, who spoke in Washington, complained that decisions about AI had been left to a handful of the richest people in the world. Steve Bannon, once a strategist in the White House, said the public could not trust tech oligarchs to regulate themselves. President Trump has dismissed fears about AI safety as a hoax. Meanwhile, Jack Clark, an Anthropic executive and co-founder, told the BBC that leaving AI as a totally unregulated industry amounted to rolling dice with immense risks, and Patrick Hillman of Logical Intelligence observed that the only institution Americans trust less than Washington these days is Silicon Valley. If you believe what you are building is dangerous, he said, show us what you are prepared to stop doing.

Altman's answer is that his industry will get it right, and that if firms cannot keep alignment and safety well ahead of their capabilities, they will slow down or stop. OpenAI, his colleague Chris Lehane said last week, is working with other laboratories, among them Anthropic and Google DeepMind, on voluntary frontier standards, with or without government support. With stakes this high, Lehane wrote, we cannot let the perfect become the enemy of the good.