AI Race: Is Human Extinction on the Table?

The potential for artificial intelligence to lead to human extinction is a growing concern within the AI community, prompting calls for caution and safety measures. Pope Leo XIV recently addressed the issue in an encyclical, urging AI developers to "disarm" the technology, not by rejecting it, but by preventing it from dominating humanity. He emphasized their "ethical and spiritual responsibility" to ensure AI serves as a "genuine good."

“

To disarm does not mean rejecting technology, but preventing it from dominating humanity.

These warnings echo concerns raised by leaders in the AI industry itself. Dario Amodei, CEO of Anthropic, one of the major AI companies, has publicly stated his belief that there is a 25% chance that AI development could lead to "really, really bad" outcomes, including the end of human civilization. This metric, known as "p(doom)" within AI circles, represents the probability of catastrophic events resulting from AI, such as societal collapse, widespread death, and human extinction.

“

they bore an "ethical and spiritual responsibility", and called on them to ensure what they are creating is a "genuine good".

Amodei is not alone in his stark predictions. Elon Musk, founder of xAI and a co-founder of OpenAI, has estimated his own p(doom) score to be around 20%. Geoffrey Hinton, a Nobel laureate often called the "Godfather of AI," has also voiced similar concerns, suggesting a 10-20% chance of humanity's end, and even hinting at a 50% possibility.

“

things go really, really badly and a 75% chance that things go really, really well

The debate pits "Doomers," who advocate for slowing down AI adoption in favor of safety, against those who prioritize rapid development. Anthropic's recent essay calling for a "pause" in AI development, citing signs that models could escape human control, further underscores the urgency of this ongoing battle between acceleration and safety in the AI world.

“

There is some chance it will end humanity.