“Does Superintelligence Scare You?” This is the title of a sister post on AIandFaith.
Perils foreseen in the advance of AI technology already frighten the Center for AI Safety. In a new report, Superintelligence Strategies, AI fear is elevated to a matter of national security. Even global security (Hendryks, Schmidt and Wang 2025).
This fear provokes two scares. The first scare is that if AGI (Artificial General Intelligence) attains a level equal to humans or even surpasses it, there is a risk that superhuman intelligence might become autonomous, rebellious, and imperious. The remedy for the first scare is to keep AI in alignment with human values and human control. Like a pet German Shepherd, keep AI on a human leashe.
The second scare is that complex AI machinery will become aligned with the goals of bad actors. Malevolent tyrants could press AI into the service of political and military tyranny. Once a global network of malevolent communications becomse established, pockets of resistance will lose leverage in communication, organization, and mobilization.
Which of the two scares frighten you the most?
The human extinction scare does not frighten Michael JD Vermer writing in Scientific American.
No scenario can be described where AI is conclusively an extinction threat to humanity. In other words, our starting hypothesis was that humans were too adaptable, too plentiful and too dispersed across the planet for AI to wipe us out using any tools hypothetically at its disposal.
According to Vermeer, AI could wipe most of us out but a remnant would survive.
Now, do you feel comforted by this? If so, let’s go on to the malevolent bad actors scenario.
AI in the Hands of Malevolent Bad Actors
Malevolent bad actors will soon have new munitions that rank with weapons of mass destruction. A military armed with an expert-level AI virologist would be able to send a novel pathogen into critical infrastructure, shutting down our energy grid, transportation, communication, and access to medical records. A cyberattack could shut down military defenses in advance of an arial attack. In brief, the very safety of civilization is vulnerable to the malicious malware of our assailants.
Up to now we have enjoyed relative tranquility because only nation-states have been capable of sophisticated attacks. This no longer obtains. Highly capable AI espionage and hacking systems can guide extremist cells from plan to execution (Hendryks, Schmidt and Wang 2025).
National Security Adopts & Confronts AI
Imagine tomorrow’s national security dilemma. If one nation successfully achieves superintelligence, it will either retain or lose control of that system. If it is able to retain control, this would likely undermine the security of its rival nations. On the other hand, if the nation loses control, renegade AI could undermine the security of all states (Katzke and Hendryks 2025).
MAIM describes the situation: Mutual Assured AI Malfunction. This would be a deterrence regime that resembles what we went through during the Cold War, namely, MAD or Mutually Assured Destruction. Now, does superintelligence scare you?
Should we resist?
Should we resist the advance of technology? No. That won’t work.
We should avoid the temptation to erect a Maginot Barrier to prevent the invasion of the future into our present. There is no stopping technological innovation. Futurists Reid Hoffman and Greg Beato are emphatic.
“As hard as it may be to accurately predict the future, it’s even harder to stop it. The world keeps changing. Simply trying to stop history by entrenching the status quo—through prohibitions, pauses, and other efforts to micro-manage who gets to do what—is not going to help us humans meet either the challenges or the opportunities that AI presents” (Hoffman and Beato 2025, 3).
Understanding - Decision - Control
Rather than resistance, I recommend u-d-c—that is Understanding, Decision, and Control. The present generation should understand future forecasts, decide which future is preferred, then take the actions to guard and guide (Peters, Futures--Human and Divine 1978).
Center for AI Safety Director Dan Hendryks and colleagues are working on a plan that includes deterrence, strategic competition, and nonproliferation (Hendryks, Schmidt and Wang 2025). If deterrence effectively kept the peace during the Cold War, might it work again?
“Deterrence in AI takes the form of Mutual Assured AI Malfunction (MAIM)—today’s counterpart to MAD—in which any state that pursues a strategic monopoly on power can expect a retaliatory response from rivals. To preserve this deterrent and constrain intent, states can expand their arsenal of cyberattacks to disable threatening AI projects. This shifts the focus from winning the race to superintelligence to deterrence” (Hendryks, Schmidt and Wang 2025).
Foresight Methodology
As the Center for AI Safety has shown us, ethical anticipation and ethical application count. First, look ahead. Make forecasts. Mariarosaria Taddeo and Luciano Floridi employ "foresight methodologies to indicate ethical risks and opportunities and prevent unwanted consequences" (Taddeo August 24 2018, 751). We need our futurists to anticipate tomorrow’s threats today.
Second, alternatively, promote a planetary vision of the common good and advocate for a prophylactic set of international guardrails. Pope Francis provides an example. He wants to ensure that AI “applications are used to promote human progress and the common good” (Francis 2025, §4). When I think of the common good, I think planetarily.
A vision of the common good for the planet, thirdly, should guide and energize international preemptive cooperation. Professor John Pittard at Yale Divinity School envisions a global alliance to address and regulate artificial intelligence on behalf of the entire planet.
The planetary common good ideal should not count as a simple alternative to Superintelligence Strategy, to be sure. Though perhaps more idealistic, it is still complementary to strategies already proposed.
Conclusion
Again: does superintelligence scare you? Well, it scares AI techies such as Elon Musk. “AI has great power to do good and evil,” wrote Musk in a Tweet. “Better the former.”
Our Jewish, Muslim, and Christian colleagues at AI and Faith are already engaged in future forecasting and lifting up positive and holistic values. The mission of AI and Faith is to equip and encourage people of faith to bring time-tested, faith-based values and wisdom to ethical AI conversation.
The two scares evoked by Superintelligence are the fear of AI takeover and the fear of AI in the hands of bad actors. Because Superintelligence has not yet arrived, we have a few breathless moments to prepare a strategy to keep advancing technology aligned with the highest of human values.
Substack H+ 2008: The Two Scares of Superintelligence
AI and Faith: Does Superintelligence Scare You?
Patheos H+ 2001: Is ChatGPT Intelligent?
Patheos H+ 2002: AI Warning: Utopia or Extinction?
Patheos H+ 2003: Your Robot Pastor is Here
Patheos H+ 2004: AI Brinkmanship
Patheos H+ 2005: Is AI a shortcut to virtue? To holiness?
Patheos H+ 2006: Creating an AI Co-Creator? Philip Hefner’s Mind
Patheos H+ 2007: Hopes and Hazards of AI
▓
Ted Peters (Ph.D., University of Chicago) is a public theologian directing traffic at the intersection of science, religion, and ethics. Peters is an emeritus professor at the Graduate Theological Union, where he co-edits the journal, Theology and Science, on behalf of the Center for Theology and the Natural Sciences, in Berkeley, California, USA. He recently co-edited Astrobiology: Science, Ethics, and Public Policy (Scrivener 2021) as well as Astrotheology: Science and Theology Meet Extraterrestrial Intelligence (Cascade 2018). He edited The Promise and Perils of AI and IA: New Technology Meets Religion, Theology, and Ethics (ATF 2025). He also co-edited Religious Transhumanism and Its Critics (Roman and Littlefield 2022) and The CRISPR Revolution in Science, Ethics, and Religion (Bloomsbury 2025). See his blogsite [https://www.patheos.com/blogs/publictheology/] and his website [TedsTimelyTake.com].
▓
References
Francis, Pope. 2025. Antiqua et Nova. Vatican: Dicastery for the Doctrine of the Faith & Dicastery for Culture and Education.
Hendryks, Dan, Eric Schmidt, and Alexandr Wang. 2025. Superintelligence Strategies. Center for AI Safety.
Hoffman, Reid, and Greg Beato. 2025. Superagency: What could possibly go right with our AI future? New York: Authors Equity.
Katzke, Corin, and Dan Hendryks. 2025. Center for AI Safety Newsletter #49: Superintelligence Strategy. Center for AI Safety.
Peters, Ted, 1978. Futures—Human and Divine. Louisville: Westminster John Knox.
Peters, Ted. 2025. The Promise and Peril of AI and IA. Adelaide: ATF.
Taddeo, Mariarosaria, and Luciano Floridi. August 24 2018. "How AI can be a force for good." Science 361:6404 751-752.
Vatican. 2002. Ethics in Internet. Pontifical Academy for Social Communications.