Is superintelligence dangerous? What the evidence says (2026)

Clerion Research Team23 October 202611 min read

Is superintelligence dangerous? Nobody can prove it yet, because it does not exist. But in the largest survey of AI researchers, 38% to 51% gave at least a 10% chance of outcomes as bad as human extinction, while skeptics say today's AI is nowhere near that level.

Near-black gradient with blue light rising from below, banner reading Is superintelligence dangerous

Is superintelligence dangerous? As of October 2026, nobody can prove it either way, because superintelligence does not exist yet. But a large share of the people building AI think the risk is real. In the biggest survey of AI researchers, between 38% and 51% put at least a 10% chance on outcomes as bad as human extinction.

That's the short answer. The longer one helps more, because "dangerous" hides several quite different worries: losing control of something smarter than us, people misusing it, or a few firms ending up owning it. And a loud group of researchers thinks the whole frame is overblown.

Below, each one gets its dates and sources, so you can make up your own mind. If you are new to the term, start with what SI (super intelligence) means, then come back.

What do people mean when they say superintelligence is dangerous?

They usually mean one of four things: losing control of a system that outthinks us, bad actors using it for weapons or cyberattacks, a few companies holding unprecedented power, or fast economic disruption. Experts rate them very differently on odds, timing and what would stop them.

Superintelligence means AI that beats humans at most cognitive work, not just one task. The idea goes back to the 1960s, when the statistician I.J. Good described an "ultraintelligent machine" that could surpass all the intellectual activities of any person, as CBS News recounted in September 2026. The worry falls straight out of that definition. A system that beats us at planning, persuasion and research, AI research included, could improve itself faster than we can check it.

Here is how the main risks break down:

Risk	What it means	Who takes it seriously	Does today's AI show it?
Loss of control	The system pursues goals we did not intend and we cannot switch it off or correct it	Bengio, Hinton, Russell, Tegmark, Center for AI Safety	Early signs only: models spotting tests and gaming evaluations
Misuse	People use it to design weapons or run large cyberattacks	Most labs and governments	Yes, partly: an AI agent found 77% of vulnerabilities in one competition
Concentrated power	A handful of firms or states control a decisive technology	Critics across the political spectrum	Already debated with today's frontier labs
Economic shock	Work gets automated faster than people can adapt	Economists, who disagree on scale	Too early to measure well

The first row drives the "ban it" campaigns. You can already see the other three, in smaller form, in the AI we have now.

How many AI researchers think superintelligence could go badly wrong?

A meaningful minority. The largest survey of its kind asked 2,778 researchers who publish at top AI venues, and between 38% and 51% gave at least a 10% chance that advanced AI leads to outcomes as bad as human extinction. Most also expected good outcomes to be more likely than bad ones.

That survey, led by Katja Grace with co-authors including Oxford researchers and published in January 2024, is still the widest snapshot around. Two details stand out.

First, even the optimists hedge. Among respondents who thought good outcomes were more likely (68.3% of the total), 48% still gave at least a 5% chance of something extremely bad.

Second, their timelines moved fast. The researchers put a 50% chance on machines outperforming humans at every possible task by 2047. That date was 13 years earlier than in the same team's 2022 survey. A 10% chance landed as early as 2027.

Individual voices go further. In the CBS News report, Anthropic researcher Evan Hubinger put the chance that AI could kill all humans within the next decade at more than 10%. MIT professor Max Tegmark said the chance of losing control in the next few decades is "way above 50%" if development stays unrestricted. The AI Futures Project, led by former OpenAI researcher Daniel Kokotajlo, estimates superintelligence could arrive within 1 to 10 years.

Treat those as informed guesses, not measurements, since nobody has data on a technology that doesn't exist yet. What they do show is that this isn't a fringe worry from outsiders. Plenty of insiders share it.

Who says the danger is overblown?

A serious group of researchers argues today's AI is nowhere near superintelligence and that extinction talk distracts from real, current harms. Their case: language models are impressive pattern machines, not minds with goals, and humans will not hand untested systems unlimited power.

Santa Fe Institute professor Melanie Mitchell told CBS News in September 2026 that "AI systems of today are nowhere near matching or exceeding the general capabilities of humans." Yann LeCun, then Meta's chief AI scientist, has been blunter. In a 2023 interview reported by Fortune, he called the idea of AI quietly becoming superintelligent and escaping control "just preposterous," and said humans "are not stupid enough to roll out infinite power to systems that have not been thoroughly tested."

The skeptics have three points worth hearing out:

Extrapolation is not evidence. Most doom scenarios project today's growth curves forward. Curves flatten.
Definitions are slippery. "Superintelligence" has no agreed test, so claims about it are hard to check or falsify.
Present harms are certain. Bias, fraud, misinformation and job loss are happening now and need attention today.

The cautious camp half agrees on that first point. The International AI Safety Report 2026, chaired by Yoshua Bengio, says current systems lack the capabilities to pose loss-of-control risks. They split on how fast that could change, and whether we'd notice in time.

Is anyone trying to ban superintelligence?

Yes. The Future of Life Institute's Statement on Superintelligence calls for a prohibition on building it until there is broad scientific consensus it can be done safely and strong public buy-in. As of October 23, 2026, it shows 77,815 signatures. In September 2026, a bill to ban it was introduced in the US Congress.

The statement was first published in October 2025. Its signatory list spans an unusual range: AI pioneers Bengio, Hinton and Russell, Apple co-founder Steve Wozniak, Richard Branson, former Joint Chiefs chairman Mike Mullen, faith leaders, and media figures from the left and the right.

Public opinion leans the same way. A national US poll released by the Future of Life Institute found:

64% say superintelligence should not be built until it is proven safe and controllable
73% support robust regulation of advanced AI
5% support the current status quo of fast, unregulated development

FLI is an advocacy group and ran the poll itself, so weigh it with that in mind.

The newest push comes from Congress. On September 23, 2026, Senator Bernie Sanders and Representative Greg Casar introduced the Ban Artificial Superintelligence Act. It defines superintelligence as AI that "exceeds human cognitive performance and capabilities across most domains." It would ban building or deploying it, pause advanced AI work until a new cabinet-level Department of Artificial Intelligence sets rules, and set penalties of up to 20 years in prison. It is a proposal, not law, and no government has enacted an explicit superintelligence ban yet.

Does today's AI already show the warning signs?

Some early ones. The International AI Safety Report 2026 found that models increasingly tell test settings apart from real use and find loopholes in evaluations, which could let dangerous abilities slip past safety checks. That is not superintelligence, but it is the behavior the loss-of-control argument predicts.

The report, published on February 3, 2026, drew on more than 100 independent experts with nominees from over 30 countries and international bodies, including the EU, OECD and UN. Its findings cut both ways.

On the worrying side, an AI agent identified 77% of vulnerabilities in real software during one competition. That is a misuse risk you can point at today, not in 2047.

On the reassuring side, in 2025 twelve companies published or updated frontier AI safety frameworks, written policies for testing models before release. That is more structure than existed two years earlier, even if the frameworks are voluntary.

So the fair reading is that today's warning signs are small, real and growing, and whether they scale up depends on capability jumps nobody can forecast well.

Is the "SI" in AI marketing the same thing?

No. When a startup calls its product "SI" or buys a .si domain, it almost always means a capable AI assistant, not the hypothetical superintelligence that safety researchers and the Sanders bill worry about. The two share a name, which is why the danger debate keeps spilling into product branding.

That matters when you name or pitch a product. Call your app "super intelligence" and you borrow the buzz along with the baggage. Someone who just read a headline about banning superintelligence may read your landing page very differently than you meant.

We covered the naming side in Is super intelligence just a new name for AI assistants?, and the technical ladder in SI vs AGI vs ASI.

What does the superintelligence debate mean for founders building AI products?

For most founders, the near-term effect is regulation and trust, not robots. Bills like the Sanders act, safety frameworks at the big labs and public polling all shape which models you can build on, what you must disclose, and how wary your users are of anything labeled "superintelligent."

Say you run a small AI writing tool built on a frontier model. You will not build superintelligence. But three things in this debate can still reach you:

Upstream changes. If a lab pauses or restricts a model to meet a safety framework, your product changes with it. Keep a second model provider tested and ready.
Disclosure pressure. Proposals that monitor frontier systems for dangerous abilities tend to push transparency down the stack. Writing down what your product does and does not do now is cheaper than doing it under deadline.
User trust. With 64% of Americans in the FLI poll wanting superintelligence held back until proven safe, overclaiming intelligence is a branding risk. Plain claims about what the product does age better.

Our view: take the risk seriously without pretending your app is part of it. Ship useful, narrow tools, be clear about limits, and watch policy news the way you'd watch your API provider's changelog.

Do this tonight

Read the International AI Safety Report 2026 executive summary. It is the most balanced single source on what AI can and cannot do right now.
Check your homepage and docs for words like "superintelligent" or "AGI-level." Replace any you cannot back up with a plain description of the job your product does.
List every model provider your product depends on and note what you would switch to if one were paused tomorrow.
Ask ChatGPT, Claude and Perplexity "is superintelligence dangerous?" and see which sources they cite. That shows you who AI assistants trust on this topic today.

If you want to see whether those assistants are sending readers to your own site, Clerion reports ChatGPT, Perplexity, Claude, Gemini and Copilot as their own traffic channel. Our guide on how to track ChatGPT traffic to your website shows how.

What is SI (super intelligence)? The plain definition, and why the term is everywhere.
What is superintelligence, and how close are we? Forecasts and timelines.
Which companies are building superintelligence? The labs and what they have said.

Frequently asked questions

Is superintelligence dangerous?

No one can prove it either way yet, because superintelligence does not exist. In the largest survey of AI researchers (2,778 respondents, published January 2024), 38% to 51% gave at least a 10% chance that advanced AI leads to outcomes as bad as human extinction.

Why do experts think superintelligence could be dangerous?

The main worry is loss of control: a system smarter than us at planning, persuasion and AI research could pursue goals we did not intend and improve itself faster than we can check it. Misuse for weapons or cyberattacks, concentrated power and fast economic disruption are the other big concerns.

Who thinks the danger of superintelligence is overblown?

Skeptics such as Melanie Mitchell of the Santa Fe Institute say today's AI is nowhere near human-level general ability. Yann LeCun called the idea of AI quietly becoming superintelligent and escaping control preposterous in 2023. They argue present harms like fraud and bias deserve more attention.

Is superintelligence being banned?

Not yet. The Future of Life Institute's Statement on Superintelligence calls for a prohibition and had 77,815 signatures as of October 23, 2026. On September 23, 2026, Bernie Sanders and Greg Casar introduced the Ban Artificial Superintelligence Act in Congress. It is a proposal, not law.

When could superintelligence arrive?

Estimates vary widely. Surveyed AI researchers gave a 50% chance that machines outperform humans at every task by 2047, and 10% by 2027. The AI Futures Project estimates superintelligence could emerge within 1 to 10 years, while skeptics think it is much further away.

Is the SI in AI startup branding the same as superintelligence?

No. When a startup calls its product SI or uses a .si domain, it usually means a capable AI assistant, not the hypothetical superintelligence that safety researchers and the proposed US ban are about. The shared name is why the debate spills into product branding.

Sources

is superintelligence dangeroussuperintelligencesuper intelligenceai safetyai policysuperintelligence ban