Glonce

Sam Altman tells UN Security Council: don't train models you can't keep under control

policyThe Glonce editorsOpenAI🔥59Glonce rating59Impact50Novelty40Relevance85Surprise35Players90Credibility90
Sam Altman tells UN Security Council: don't train models you can't keep under control

OpenAI CEO Sam Altman addressed the United Nations Security Council on artificial intelligence, framing a choice between AI as a 'new Renaissance of creativity and discovery' and AI as a 'new Industrial Revolution of upheaval and disarray.' He said he largely agreed with Professor Bengio, who had outlined ways AI progress could go badly, though Altman narrowed the list to two. The first risk is losing control of the future to AI, particularly as systems approach the ability to improve themselves and future versions of themselves (recursive self-improvement), which could accelerate progress beyond what institutions can follow or intervene in. Altman said the industry must not accept excessive technological risk just because competitive pressure makes slowing down feel costly, and stated that OpenAI has 'unilaterally slowed down in the past' and will do so again. He argued companies should avoid building systems for which they cannot make strong alignment, monitorability and safety guarantees, adding that it does not matter whether a catastrophe risk is estimated at 10%, 1%, 12% or .1%: 'none of these levels are remotely acceptable,' and models should not be trained without an extremely strong case they can be kept under human control. The second risk he described is concentration of power: no single person, company or country should be able to use the most powerful AI models to impose its worldview on everyone else, and a belief that only one actor can be trusted with the technology should not be used to justify anything. Altman said OpenAI tries to walk a 'middle path' between blind optimism and doomerism, guided by three principles: people must remain at the center of AI decision-making, with alignment meant to keep systems under human control and reflecting human values rather than automating human judgment; the benefits of scientific and economic progress must be by and for people, helping researchers, doctors, teachers and entrepreneurs; and the technology must empower people individually rather than positioning OpenAI as 'the heroes of this story.' He cited a progression in AI mathematical capability: models that were merely okay at grade-school math three summers ago reached gold-medal level at 'the most prestigious international math competition' last summer, and a few weeks before the speech, one of OpenAI's models solved one of the Millennium Prize Problems, the Navier-Stokes equations, which are used in aircraft design, weather forecasting and blood-flow research. Altman said the most important AI decisions cannot be made by labs in San Francisco alone but must be shaped through democratic processes and governments accountable to their people, and called for international cooperation on complementary national and international frontier AI standards: shared measures for capabilities, risk assessment and safeguards; common standards so countries can compare evidence and verify compliance; fast incident reporting and classification protocols; and secure channels among governments, infrastructure operators and technical experts to share emerging threats. He said such standards must not lock in incumbents or favor one business model, and must support both open and closed model developers and both new entrants and established labs, while each government decides how to incorporate them into its own legal system. He closed by describing a crossroads between a future where power is concentrated and decisions are opaque, and one where AI is guided by democratic institutions and expands people's ability to live as they choose, calling the latter 'the future we believe is worth building.'

Key facts

  • Sam Altman addressed the UN Security Council, describing two ways AI progress could go badly: losing control of the future to AI (especially via recursive self-improvement) and AI concentrating power in too few hands.
  • Altman said OpenAI has 'unilaterally slowed down in the past' and will do so again, and that models should not be trained without an extremely strong case they can be kept under human control.
  • He said catastrophe-risk estimates of 10%, 1%, 12% or .1% are all 'remotely unacceptable,' regardless of the exact figure.
  • OpenAI reached gold-medal level at the top international math competition last summer and, a few weeks before the speech, one of its models solved the Navier-Stokes equations, one of the Millennium Prize Problems.
  • Altman called for international frontier AI standards covering capability measurement, risk assessment, incident reporting and threat-sharing channels, saying standards must not favor incumbents or one business model over another.

Why it matters

This is one of the most prominent venues in which an AI lab leader has directly addressed global governance of frontier AI: a sitting Security Council session rather than an industry conference or a company blog aimed at developers. Altman used it to state explicit red lines (no training without strong control guarantees, no single actor imposing its worldview via AI) and to publicly claim OpenAI has already slowed itself down before.

Who it affects

The remarks are aimed at national governments and the Security Council as a body, but they also bear on AI labs, competitors and the broader public: Altman frames the choice between concentrated, opaque AI power and a democratically shaped alternative as one that affects everyone, not just the companies building the technology.

How to use it

There is no product, price or release here: it is a policy speech. Readers tracking AI governance can use it as a marker of where OpenAI's leadership publicly positions the company on safety commitments and international standards, useful for comparing against OpenAI's future actions.

How solid is it

The text is OpenAI's own published transcript of Altman's remarks, so the quotes and claims are primary-source and verified against that transcript. It does not independently confirm how Security Council members reacted, and details such as the exact date, which country held the Council presidency, or which specific model solved the Navier-Stokes problem are not given in the source.

Risks and caveats

The catastrophe-risk percentages Altman cites (10%, 1%, 12%, .1%) are rhetorical examples to argue no risk level is acceptable, not an actual risk estimate for any real system. His claim that OpenAI has 'unilaterally slowed down' AI development in the past is asserted without specifics on when or how, and the speech is a policy statement of intent rather than a binding commitment or new regulation.

“It doesn't matter whether people put the risk of catastrophe at 10%, or 1%, or 12%, or .1%. None of these levels are remotely acceptable. And we should not train models that we cannot make an extremely strong case that we will be able to keep under human control.”

— Sam Altman, OpenAI CEO