Pope Francis warns of 'new atrocities' from unregulated AI

Pope Francis has issued a stark warning about the unchecked advancement of artificial intelligence, comparing its potential dangers to the societal upheaval caused by the Industrial Revolution. In his first encyclical, "Magnifica Humanitas," the pontiff, who is the first mathematically trained Pope, directly confronts the profound changes technology is inflicting on humanity.

Echoing Pope Leo XIII's 1891 encyclical "Rerum Novarum," which addressed the plight of workers during the Industrial Revolution, Pope Francis courageously speaks out on issues many Western political leaders avoid. He cautions that artificial intelligence, if left unregulated and concentrated in the hands of a select technological elite, could result in "new atrocities." This is not an exaggeration, the encyclical asserts, but a rigorous diagnosis of the current technological landscape.

The encyclical critiques the nature of digital surveillance, contrasting it with Jeremy Bentham's 18th-century panopticon prison concept. While Bentham's design involved a single, unseen guard observing prisoners, modern digital surveillance is inverted. Individuals voluntarily place themselves under the gaze of numerous observers, sharing personal data, locations, and intimate details with enthusiasm, often with pride. This digital panopticon is far more pervasive and insidious because it operates on voluntary participation rather than coercion.

This voluntary participation is the unique characteristic of digital surveillance, making it the most effective form of social control ever devised. It eliminates the need for a visible guard, replacing force with seduction and turning the observed into the primary agent of their own monitoring. Economist Shoshana Zuboff's concept of "surveillance capitalism" is highlighted, describing an economic model that extracts human behavioral data as free raw material. This data is then transformed into predictive products sold in behavioral futures markets, meaning users are the product, not the customers, of these digital platforms.

Every search, click, pause, movement, and expressed emotion is captured, analyzed, and monetized. The implications extend beyond individual privacy to the very autonomy of human behavior. The encyclical warns that if behaviors can be predicted with sufficient accuracy, they can also be manipulated, blurring the line between influence and outright manipulation. Pope Francis alerts that "small, highly influential groups can shape information and consumption, conditioning..." the minds of individuals.