OpenAI CEO Sam Altman says the public is justified in worrying that artificial intelligence could become dangerously powerful or give a small number of companies excessive influence over society. His proposed response, however, creates a difficult governance question of its own: how much should the world rely on the companies building the most capable AI systems to determine whether those systems are safe?
Speaking at Salesforce’s Dreamforce conference in San Francisco on September 15, Altman identified two broad dangers as frontier AI advances. One is that humanity could lose control of increasingly capable systems; the other is that companies developing those systems could accumulate excessive economic and social power. He said the world is “right to be afraid” of those possibilities while arguing that AI developers should be trusted to make responsible decisions because they understand the magnitude of what they are building.
The remarks arrived during an unusually intense period for AI safety. Executives and researchers across several leading AI laboratories have recently called for stronger safeguards, slower development under dangerous conditions and greater external scrutiny as frontier models gain more autonomous capabilities. The debate is no longer simply whether advanced AI carries risks, but who gets to decide when those risks have become unacceptable.
Altman Identified Two Different AI Risks
The distinction between the two risks Altman described is important. The first concerns the technology itself: increasingly autonomous AI systems could behave in unexpected ways, evade intended controls or become capable enough that failures have consequences beyond the ability of developers to contain them.
The second concerns human institutions rather than machine behavior. A small number of companies could control systems capable of influencing employment, information, scientific research, software development and other important parts of the economy. Even if those systems remained technically aligned with their developers’ intentions, concentration of that much capability could create questions about whose values the models reflect and who has the authority to make consequential decisions about their deployment.
Altman acknowledged that concern directly at Dreamforce, saying people have legitimate reasons to fear AI developers accumulating too much power and potentially exerting undue influence on the economy or promoting particular worldviews. That makes his argument for public trust especially significant because the trust question is itself part of the concentration-of-power problem he described.
“Trust Us” Is Different From Independent Verification
There is an important difference between believing that AI executives sincerely want to prevent catastrophic failures and designing a governance system that does not depend on their intentions. Industries involving aviation, pharmaceuticals, nuclear energy and financial markets generally combine corporate responsibility with external rules, inspections, reporting obligations and independent institutions.
AI presents a harder version of that problem because technical capabilities can change rapidly. A regulation written around one generation of models may become outdated as systems gain new abilities, while external regulators may struggle to recruit enough technical expertise to evaluate models being developed inside heavily resourced private laboratories.
That does not make independent oversight unnecessary. It makes the design of oversight more difficult. Governments and independent evaluators need sufficient technical access to determine whether safety claims are supported by evidence without requiring companies to publicly disclose model details that could create security risks or expose valuable intellectual property.
The central governance question is therefore not whether OpenAI or other laboratories should be trusted at all. It is which decisions can reasonably remain internal and which decisions require independent verification because the potential consequences extend beyond the company making them.
OpenAI’s Own Governance Framework Recognizes That Problem
OpenAI has already moved beyond a purely trust-based safety model in its formal policies. Its Frontier Governance Framework describes risk assessment and mitigation processes covering cybersecurity, chemical and biological risks, harmful manipulation and loss of control, alongside incident response, security management, model reporting and external expert involvement.
The company’s Preparedness Framework also establishes capability thresholds intended to trigger stronger safeguards. Models reaching sufficiently advanced levels in designated risk categories face additional requirements before deployment, while systems reaching critical capability thresholds can require stronger protections even during development.
Those mechanisms matter because they attempt to turn broad safety principles into operational decisions. Instead of simply saying a model should be safe, developers need evaluations capable of identifying dangerous capabilities, controls designed to reduce those risks and governance procedures for deciding whether the remaining risk is acceptable.
OpenAI’s frameworks are nevertheless largely designed and implemented by OpenAI. External experts can contribute to evaluation and oversight, but the broader question remains whether voluntary corporate frameworks provide enough accountability as the economic and security significance of frontier models increases.
OpenAI Says AI Should Be Democratically Governed
That tension becomes clearer when compared with OpenAI’s own recent statements about public participation. In September, the company argued that artificial general intelligence should ultimately be democratically governed and that people need enough information about frontier capabilities, risks and safeguards to have a meaningful voice in how advanced AI develops.
That principle points beyond simple corporate trust. Democratic governance requires mechanisms through which governments, researchers, civil society and the public can scrutinize consequential decisions rather than relying entirely on assurances from executives. Transparency does not necessarily mean releasing sensitive model weights or security information, but it does require enough evidence for outsiders to evaluate important safety claims.
OpenAI has increasingly published system cards, preparedness assessments and technical safety material around major model releases. Such disclosures can improve accountability by giving researchers and policymakers more information about known capabilities and limitations.
The harder question is what happens when an external evaluator disagrees with a company’s decision to proceed. Transparency can reveal a dispute, but only governance determines who has authority to resolve it.
Frontier AI Has Crossed New Cybersecurity Thresholds
The safety debate has become more urgent because frontier models are gaining capabilities with direct security consequences. In 2026, OpenAI reported that its Astra model had reached what its Preparedness Framework defines as a Critical cybersecurity capability threshold. According to the company, such capability can include finding previously unknown vulnerabilities and developing ways to exploit hardened systems with substantially less human guidance.
OpenAI responded by introducing stronger controls around model development and testing, including isolated environments, restricted network and tool access, stronger model-weight protections and additional monitoring. It also said certain internal activities involving Astra were paused until strengthened security requirements could be satisfied.
Those actions demonstrate why discussions about slowing AI development are no longer entirely theoretical. If a model’s capabilities advance faster than the systems used to contain and monitor it, a laboratory may need to delay some work while safeguards catch up.
They also demonstrate why independent evaluation becomes valuable. The more consequential the capability threshold, the greater the public interest in knowing whether the organization developing the system has measured the risk accurately and whether the safeguards actually work under realistic conditions.
The AI Industry Is Moving Toward Shared Safety Standards
Altman’s comments also come amid signs that competing AI laboratories are considering greater coordination around frontier safety. Recent discussions involving leaders from OpenAI, Anthropic and other AI developers have focused on slowing development when necessary, using independent evaluations and creating shared standards for increasingly powerful systems.
Coordination could address a serious competitive problem. A company may believe additional safety testing is necessary but worry that delaying a release will allow a competitor to gain market share or technical leadership. If every frontier laboratory faces the same incentives, competition can push the entire industry toward moving faster than individual executives consider prudent.
Shared minimum standards could reduce that pressure by establishing safety requirements that apply across companies. Independent evaluations could similarly make it harder for individual laboratories to lower their own thresholds simply because competitors are advancing rapidly.
But voluntary agreements have limitations. Companies can change policies, interpret standards differently or withdraw from commitments when commercial incentives shift. That is why the relationship between industry standards and enforceable government rules is likely to become one of the central AI-policy debates of the next several years.
AI Safety Is Also a Competition Problem
Frontier AI development requires enormous amounts of computing infrastructure, specialized chips, data, engineering talent and capital. Those requirements naturally favor companies capable of raising billions of dollars and operating enormous computing clusters. If the cost of training leading systems continues increasing, the number of organizations able to compete at the frontier could remain limited.
Safety requirements can reinforce that concentration if only the largest companies can afford sophisticated evaluations, security infrastructure and compliance teams. Regulation designed to constrain powerful laboratories could therefore unintentionally make it harder for smaller competitors to enter the market.
The opposite problem is equally serious. Weak requirements could allow companies to race toward increasingly capable systems without adequate safeguards simply because no developer wants to fall behind. Policymakers therefore face a difficult balance between preventing unsafe competition and creating rules that entrench the existing market leaders.
Altman’s warning about concentrated corporate power is relevant precisely because OpenAI is itself one of those frontier companies. The governance system needs to manage technological risk without making a handful of existing laboratories the permanent gatekeepers of advanced AI.
Corporate Incentives Help, but They Are Not the Same as Accountability
AI companies have substantial reasons to avoid catastrophic failures. A serious incident could harm users, destroy corporate value, trigger regulatory intervention and damage public confidence in the technology. Developers therefore have genuine commercial as well as ethical incentives to invest in safety.
Those incentives, however, do not always align perfectly with the public interest. Companies also face pressure from investors, competitors, customers and employees to release stronger products quickly. A model that creates a small probability of an extremely severe failure may still appear commercially attractive if the risk is difficult to measure and competitors are moving ahead.
That is a familiar reason societies create independent oversight. Regulation does not necessarily assume that companies are acting maliciously; it recognizes that organizations make decisions within incentive structures that may not capture every cost imposed on the wider public.
The question surrounding frontier AI is how to build such oversight without freezing technical progress or giving regulators authority over systems they lack the expertise to evaluate.
Transparency Becomes More Important as Models Gain Autonomy
As AI systems become capable of performing longer sequences of actions, monitoring what they do becomes more important. A chatbot that generates a wrong paragraph creates a different class of risk from an autonomous agent capable of writing software, operating tools, interacting with external systems and pursuing a complex objective over time.
Developers are consequently investing in monitoring systems intended to detect dangerous or misaligned behavior while models operate. Red-team exercises, capability evaluations and controlled testing environments can expose failures before broad deployment.
No evaluation regime can guarantee that every dangerous behavior will be discovered beforehand. Models may behave differently in real-world environments, encounter situations absent from testing or discover strategies evaluators did not anticipate.
That uncertainty strengthens the case for incident reporting and post-deployment monitoring. Aviation safety improved not because engineers learned to guarantee that accidents could never happen, but because incidents produced investigations, shared evidence and changes intended to prevent recurrence. Advanced AI may require a comparable culture of systematic learning, although the institutions responsible for it are still being built.
The Public Debate Should Move Beyond “Trust or Don’t Trust”
Altman’s remarks can easily be reduced to a provocative contradiction: the public should be afraid of powerful AI companies but should also trust them. That framing captures attention, but it does not resolve the underlying policy problem.
Society routinely relies on organizations operating dangerous or economically important technologies without giving them unlimited discretion. Trust is supported by standards, evidence, audits, legal responsibilities, independent expertise and mechanisms for investigating failures. AI governance can develop along similar lines even if the technical details differ substantially from older industries.
The relevant question is therefore not whether Sam Altman, OpenAI or any other AI executive personally deserves public confidence. It is whether the institutions surrounding frontier AI are strong enough that safety does not depend primarily on confidence in particular executives.
That requires companies to demonstrate how they evaluate dangerous capabilities, explain the thresholds that would cause them to delay development or deployment, provide appropriate access to independent evaluators and disclose serious incidents when they occur.
Altman’s Warning Shows How Much the AI Debate Has Changed
Only a few years ago, much of the public AI debate centered on whether generative models would become economically useful. The discussion has shifted rapidly as systems gained stronger reasoning, coding, tool-use and autonomous capabilities. Leaders of frontier laboratories are now openly discussing loss of control, dangerous cybersecurity capabilities and circumstances in which development might need to slow.
That shift should not automatically be interpreted as proof that catastrophic AI outcomes are inevitable. Companies developing frontier systems have incentives to emphasize both the capabilities and risks of their technology, and predictions about future systems remain uncertain.
But dismissing the concerns simply because they come from industry leaders would be equally unhelpful. OpenAI’s own capability assessments show that concrete security thresholds are already changing as models improve, requiring stronger safeguards during development and deployment.
The useful response is to demand evidence: what can the systems actually do, how reliably can those capabilities be controlled, what happens when safeguards fail, and who independently verifies the answers?
Trust in Frontier AI Will Have to Be Earned Through Institutions
Altman is right about one fundamental point: increasingly capable AI creates questions important enough that the public should take them seriously. Loss of control and excessive concentration of technological power represent different risks, but both become more consequential as AI systems gain greater autonomy and economic importance.
Where his argument becomes more difficult is the idea that confidence in AI companies themselves can carry most of the burden. Corporate safety frameworks are important, and OpenAI has built increasingly detailed mechanisms for evaluating frontier risks. Yet the company’s own argument that advanced AI should ultimately be democratically governed points toward a broader model of accountability.
The next phase of AI governance will therefore be about turning voluntary safeguards into institutions that can be independently examined and, where necessary, enforced. Industry expertise will be indispensable because frontier laboratories understand their systems better than almost anyone outside them. That expertise should inform oversight rather than substitute for it.
The public does not have to choose between blindly trusting AI companies and assuming they cannot be trusted at all. A more durable approach is to build a system in which consequential safety claims can be tested, important incidents are disclosed, independent experts have meaningful access and no single company gets to decide by itself how much technological risk the rest of society should accept.
