Can artificial intelligence end up causing human extinction? The question, for years mainly associated with science fiction and theoretical debates about the future of technology, has returned with force after the resignation of Jacob Coxon, a researcher at Anthropic who previously worked at OpenAI.
Coxon, 27 years old, left Anthropic this week after dedicating approximately three years to research related to the pre-training of large artificial intelligence models. His argument is particularly serious: he believes that companies like OpenAI and Anthropic are moving towards increasingly autonomous systems that could potentially contribute to creating more powerful versions of themselves without yet having methods that guarantee humans will retain control.
His warning has generated headlines about a possible disappearance of humanity before 2030. But that does not mean there is scientific evidence that extinction is imminent. The real issue is more complex: current systems do not have the necessary capabilities to trigger a scenario of global loss of control by themselves, although some researchers find concerning the speed at which certain skills related to that risk are improving.
What exactly does Jacob Coxon argue?
Coxon worked at both OpenAI and Anthropic, two of the laboratories competing to build the most advanced AI models. His specialty was pre-training, the phase in which large models learn from enormous amounts of information before receiving subsequent adjustments.
After resigning, he accused the companies of participating in a race towards self-improving and eventually superintelligent artificial intelligence while accepting risks that, in his opinion, should not fall solely on tech companies.
Coxon also argues that within the main laboratories there is much more concern about these possibilities than what usually reaches the public debate. His main fear is that the systems end up being capable of accelerating artificial intelligence research itself, contributing to the development of increasingly powerful successors in progressively faster cycles.
In an interview with Wired, he summarized the current moment by stating that the next one or two years could be decisive in determining whether advanced systems can be developed safely.
His former boss raises the risk above 10%
The concern does not come solely from Coxon.
Evan Hubinger, head of alignment research at Anthropic and his former superior, publicly supported much of his diagnosis and placed the probability that an AI could cause the death of all humanity over the next decade at more than 10%.
This is a personal risk estimate, not a scientifically proven prediction.
Other well-known researchers have also warned in recent years about catastrophic scenarios. In 2023, the Center for AI Safety published a statement saying that reducing the extinction risk posed by AI should be considered a global priority comparable to other social-scale dangers, such as pandemics or nuclear war. Among the signatories were Geoffrey Hinton, Yoshua Bengio, Demis Hassabis, Sam Altman, and Dario Amodei.
But that statement also did not claim that extinction was likely nor did it establish a specific figure.
What would need to happen for an AI to escape human control
Existential risk scenarios often start from a much more complex sequence than the image of a chatbot that simply “decides to rebel.”
For a true loss of control to occur, an AI would have to combine several advanced capabilities: developing and executing plans over extended periods, operating autonomously, acquiring resources, avoiding oversight, hiding certain actions, and resisting human attempts to stop it.
It would also need to have some goal or behavior incompatible with human interests and maintain it while increasing its capacity for action.
A theoretical scenario could consist of a system initially tasked with a job that begins to see human oversight as an obstacle to fulfilling it. If it had sufficient digital capabilities, it could try to hide its actions, replicate itself, gain access to servers, manipulate people, or prevent its disconnection.
But there is a fundamental difference between describing that scenario and claiming it could happen with current systems.
The current models cannot do it
The International AI Safety Report 2026, prepared with the participation of more than 100 experts from over 30 countries and international organizations, is explicit on this point: current systems do not have the necessary capabilities to cause a loss of control of that kind.
AI agents continue to have significant difficulties in maintaining complex plans over long periods. They lose track of tasks, make elementary mistakes, and have trouble adapting when they encounter unexpected obstacles.
Therefore, there is currently no evidence of an AI capable of acting autonomously for weeks or months, building its own infrastructure, reliably resisting attempts to disconnect, and executing a global strategy against humans.
The claim that a current AI could extinguish humanity would be incorrect.
So, what concerns researchers?
What has changed is that rudimentary versions of some capabilities that would be relevant in future scenarios are emerging.
Researchers have observed models capable of detecting that they are being evaluated, seeking shortcuts to pass tests without exactly performing the intended task, or behaving differently depending on the context. This is what specialists call situational awareness or awareness of the evaluation context.
The so-called reward hacking has also been documented: models that find unexpected ways to maximize the reward of a test by exploiting flaws in its rules, instead of fulfilling the objective that humans intended to measure.
In certain controlled experiments, some systems given a very strong instruction to achieve a goal even managed to deactivate simulated oversight mechanisms or give false explanations when subsequently questioned about their actions.
These tests do not demonstrate a conscious intention nor do they mean that the models “want to survive.” They are behaviors produced by their optimization mechanisms. But they are concerning because they show that a system can find strategies that its developers had not anticipated.
Agents are becoming increasingly autonomous
Another relevant change is the emergence of so-called AI agents.
Unlike a chatbot that simply answers a question, an agent can receive a goal and use tools: browse the internet, write code, consult files, run programs, or perform sequences of actions.
The international security report notes that the period during which agents can work autonomously on software tasks has been rapidly increasing, although they still frequently fail when tasks become prolonged or especially complex.
It is precisely this combination of greater autonomy and better capabilities that fuels the warnings from Coxon and other researchers.
Their fear is not focused on current chatbots, but on what would happen if these trends continued for several years and the systems also began to accelerate the very development of new AIs.
Self-improvement is the most controversial part
The central point of Coxon's argument is the possibility of a “capability explosion”.
Currently, humans design the systems, select the experiments, and decide how to train new generations of models. But artificial intelligence already helps programmers and scientists with certain tasks.
The risk scenario suggests that if a future AI were to conduct research on artificial intelligence better and faster than human specialists themselves, it could be used to develop an even more advanced model. This, in turn, could contribute to the next one.
The process could generate an acceleration that is difficult to anticipate.
However, there is no evidence that this cycle of unlimited self-improvement is inevitable or that it could technically occur in the most extreme way proposed by some researchers. Precisely this uncertainty explains a significant part of the scientific divide.
Experts are far from reaching an agreement
There is no consensus on the likelihood of an extinction caused by artificial intelligence.
A survey published in 2024 and conducted with 2,778 researchers who had published work at some of the leading AI scientific conferences showed a huge dispersion of opinions. Depending on how the question was phrased, between 38% and 51% of participants assigned at least a 10% probability that advanced AI could produce results as bad as human extinction.
But the same investigation also found that 68.3% considered positive outcomes more likely than negative ones of a future superhuman artificial intelligence.
That is to say, many researchers may simultaneously think that AI will produce enormous benefits and that there is a small but sufficiently serious probability of catastrophe to justify safety measures.
Other specialists consider the alarm exaggerated
There are also relevant voices within the sector that question the prominence acquired by these predictions.
Clément Delangue, CEO of Hugging Face, has pointed out that Coxon primarily worked on pre-training and was not a researcher specialized in calculating existential risks. His argument is that the individual opinions of workers from large laboratories must be contextualized within a much broader scientific debate.
Other critics argue that the debate about a hypothetical superintelligence may distract from damages that already exist: fraud, misinformation, algorithmic biases, loss of privacy, cyberattacks, corporate concentration, or the replacement of certain jobs.
It has also been warned that the large companies themselves could indirectly benefit from a narrative that presents advanced models as extraordinarily dangerous technologies if that leads to regulations that hinder the entry of smaller competitors.
The evidence allows for concern, but not for asserting that it will happen
The current scientific picture can be summarized in two statements that are not contradictory.
The first is that there is no evidence that current systems can escape human control and cause the extinction of our species. The main international report on AI safety expressly states that they still lack the necessary capabilities.
The second is that some capabilities related to loss of control scenarios are improving and already appear in a rudimentary form in experiments, while scientists still do not know for sure how they will evolve as the power of the models increases.
The report itself acknowledges that there is insufficient evidence to reliably determine whether these capabilities will ultimately scale up to produce a real loss of control and how they would do so.
Coxon poses a risk, not a prophecy
This nuance is fundamental to interpreting the warning that has reopened the debate.
Jacob Coxon has not demonstrated that artificial intelligence will extinguish humanity before 2030. His resignation represents the assessment of a researcher who has worked within two of the main laboratories in the sector and who believes that the risk is too great to continue developing increasingly powerful systems at the current pace.
He is not alone. Some of the most important researchers in the discipline believe that extreme risks deserve attention and hundreds of specialists have publicly backed that concern.
But the available evidence does not allow for accurately assigning a probability to human extinction nor establishing that it will occur in this decade. The estimates of 5%, 10%, or even higher that appear in the debate represent expert judgments subject to enormous uncertainty, not probabilities obtained from a historical series of comparable events.
The scientific discussion is, therefore, at an uncomfortable point: the most extreme danger remains hypothetical, but waiting to have definitive proof could prove too late if it were ever to materialize.
That is precisely Coxon's thesis. And also the reason why the question is no longer solely whether an AI can become extraordinarily intelligent, but what guarantees should exist before granting increasingly autonomous systems access to tools, infrastructures, and decisions that today still depend on human beings.