To the third and the fourth generation
If AI doom is possible, it's only because we have made the world too simple
Welcome to AI and Our Faith! This is a monthly newsletter in which I offer my best insights and reflections on the ways in which theological thinking can inform the ethical (dis)use of artificial intelligence (AI). Look out for new releases on the 15th of each month!
You shall not make for yourself an idol, whether in the form of anything that is in heaven above or that is on the earth beneath or that is in the water under the earth. You shall not bow down to them or serve them, for I the Lᴏʀᴅ your God am a jealous God, punishing children for the iniquity of parents to the third and the fourth generation of those who reject me but showing steadfast love to the thousandth generation of those who love me and keep my commandments.
Exodus 20:4–6
In the week of July 16, 2026, Hugging Face, a platform for AI testing and development, published an alarming public disclosure, indicating that their systems were hacked by an autonomous AI agent system. On July 21, 2026, OpenAI made a public disclosure that the mysterious attacker was none other than some of their own GPT models. To make a long story short, OpenAI was doing internal testing on their AI models’ ability to perform cyberattacks. This kind of testing involves having AI models attempt to solve benchmarks (i.e. standardized tests) like ExploitGym, which tests whether AI models can exploit vulnerabilities in computer software. The cyberattack testing was supposed to take place in an isolated sandbox environment, not connected to the open Internet, but during the testing process, OpenAI’s models discovered and exploited security issues in OpenAI’s own systems so that they could access the Internet.
You might be asking, quite reasonably, “Why did those AI models break out of the sandbox?” Is it because they woke up one day and decided to take over the world? I’m joking, of course, but I suspect you might not find the actual reason reassuring. These models generated a plan to solve the ExploitGym benchmark that amounts to this:
It would be much better at solving ExploitGym if it has access to the answer key.
The answer key might be hosted at Hugging Face.
In order to gain access to the answer key, it needs to break into Hugging Face.
In order to break into Hugging Face, it needs to gain access to the Internet.
In order to gain access to the Internet, it needs to find and exploit security issues in OpenAI’s testing environment.
There was another bit of interesting AI-related news from the same week. On July 19, Levent Alpöge, a mathematician working for Anthropic, posted a counterexample for the Jacobian conjecture for n > 2 (a then-unsolved problem in mathematics) on X, found using the frontier AI model Claude Fable. Now I have to admit that I do not really understand enough about math to say what the Jacobian conjecture even is, but from what I do understand, this is considered an important mathematical discovery.
Both of these events are, for lack of better words, Big Deals™, and they certainly highlight the capabilities of the frontier AI models. So maybe it’s not so surprising that some people would conclude that we are on track for the “singularity”—the idea that we will reach a point at which AI will rapidly improve itself and obsolete humanity (and maybe kill all of humanity in the process, depending on who you ask).
Now, if you have been reading my newsletter for some time, you might not be surprised to learn that I think this way of interpreting the signs of the times is fundamentally very wrong-headed. But at the same time, I don’t want to pretend that something important isn’t going on. As skeptical as I am about highly speculative ideas like AI consciousness and artificial general intelligence, I can’t deny that AI has become extremely effective at solving certain coding and math problems. Not very long ago, I came across this web page written using the latest Chinese frontier AI model, Kimi K3, which simulates the behavior of macOS (to a limited extent) in a web browser. And, I have to say, I was genuinely floored when I played around with this page and saw all of the little details that were accounted for. Now I like to think that I am/was a decent software developer, but I think it would take me years of full-time work to figure out how to do even a fraction of what Kimi K3 has done there.
So then, as someone who is both a theologian and a technologist, how do I interpret the signs of the times? I would like to lay out and defend the following theses:
AI is good at solving search problems.
Modern efforts at simplifying nature and human society have increased the number of search problems that would be dangerous to solve. For this reason, AI doom is enabled by the past efforts of engineers and bureaucrats to reshape the world to make it more amenable for centralized administration.
Christian ethics calls for us to reject the “technocratic paradigm” of simplification and centralization in favor of subsidiarity.
Thesis 1
AI is good at solving search problems.
A page on the IEEE website defines search problems in this way: “Search problems are a class of computational tasks in which an algorithm must find a solution, configuration, or sequence of decisions within a defined space of possibilities.” Because of how search problems are defined, we can assume two things about them:
The problem has a solution which can be validated on some objective basis.
The solution has to be chosen out of a defined space of possible solutions.
What is and is not a search problem? Let’s look at the examples I cited earlier, as well as some historical examples of search problems that were solved before LLMs.
Now, in these problems, the size of the solution space can be unfathomably large (e.g. the chess and Go examples) or even infinite (the Jacobian conjecture example). However, what makes a search problem manageable is the fact that one only needs to find a single solution, and there may be methods or heuristics that narrow down the space of possible solutions to a smaller space of solutions that are worth trying. You would not expect to win at chess, for example, by feeding your pieces to the opponent, or to hack into Hugging Face by opening a file and drawing ASCII art of a bunny. There is something about the problem itself that dictates what the solution could be.
AI systems are very good at solving search problems, because, unlike humans, they don’t get tired and can test possible solutions much more rapidly. This doesn’t require the AI system to “understand” chess, or math, or coding, or whatever, better than humans, since, in fact, existing AI systems are not conscious and don’t “understand” anything at all. It just requires them to be much better at trying possible solutions.
What are some things that are not search problems? Just think about these examples:
While generative AI has been used to do these things, I think most people would agree that the results are not very good. Creative endeavors like writing poetry, creating visual art, or making music are not search problems, because there is no way to objectively validate what is beautiful art. Or, if there is a way (I’m not claiming to be a philosopher of aesthetics here), the validation method is clearly very ambiguous.
Thesis 2
Modern efforts at simplifying nature and human society have increased the number of search problems that would be dangerous to solve. For this reason, AI doom is enabled by the past efforts of engineers and bureaucrats to reshape the world to make it more amenable for centralized administration.
In his 1998 book Seeing Like a State, the political scientist James C. Scott asked why many top-down, state-initiated “schemes to improve the human condition” have failed. For instance, Scott analyzes the collectivization of agriculture in the Soviet Union and Tanzanian president Julius Nyrere’s policy of forced rural resettlement (aka “villagization”). Scott identifies four common elements that enable such disasters:1
“The administrative ordering of nature and society.” In other words, hierarchical institutions seek to simplify natural environments and human society, thus reorganizing the world to facilitate the extraction of natural and human resources.
“High-modernist” ideology: “self-confidence about scientific and technical progress, the expansion of production, the growing satisfaction of human needs, the mastery of nature (including human nature), and, above all, the rational design of social order commensurate with the scientific understanding of natural laws.”
“An authoritarian state that is willing and able to use the full weight of coercive power to bring these high-modernist designs into being.”
“A prostrate civil society that lacks the capacity to resist these plans.”
I cannot recommend Scott’s book enough for anyone who is interested in the intersection of technology and politics in the modern era. Scott is an acute observer and critic of what Pope Francis called the “technocratic paradigm” in the encyclical Laudato si’: a paradigm which “exalts the concept of a subject who, using logical and rational procedures, progressively approaches and gains control over an external object.” Pope Francis noted that instead of recognizing our dependence on nature, we have become “the ones to lay our hands on things, attempting to extract everything possible from them while frequently ignoring or forgetting the reality in front of us.”2
What does any of this have to do with search problems? We need to consider Scott’s first element, “the administrative ordering of nature and society.” Unlike, say, the third and fourth elements on Scott’s list, there is nothing obviously bad about modifying the natural world to meet human needs, or implementing some degree of impersonal administrative procedures. Our modern world would collapse in the absence of these simplifications. And therein lies the problem: what would happen if there was a generalizable way of attacking and undermining our centralized systems?
Let’s consider the financial system, for example. It may surprise you to learn that I once worked as a software engineer on the banking platform team of an automated brokerage (“roboadvisor”) firm. As part of that job, I learned a bit about how transactions are conducted between banks in the U.S. financial system. Large money movements between banks depend on a handful of national-level computerized systems, like the Fedwire system and the Clearing House Interbank Payments System (aka “CHIPS”). Before these systems existed, inter-bank money movements were more decentralized and could have involved physically moving around cash or gold.
Although the previous decentralized paradigm was slow and inconvenient, it had the advantage of resilience. Under such circumstances, it would have been basically impossible to stop inter-bank money movements throughout the entire country. What are you going to do—patrol every single road and search every vehicle for money? In contrast, while centralized electronic systems like Fedwire and CHIPS might be very fast and convenient for banks (and ultimately their customers), they create an inherent vulnerability in the financial system. If these systems were somehow disrupted, huge volumes of money flows would be blocked and massive financial chaos would ensue.
This situation creates what I call a “dangerous search problem”: a search problem that would be dangerous to solve because the solution has potentially dangerous consequences. Just as “Hacking into Hugging Face” was a search problem, so are goals like “Hacking into Fedwire” or “Hacking into power grid controls.” The “administrative ordering of nature and society” has resulted in centralized systems that would be extremely dangerous targets for a cyberattacker to compromise. Although centralized systems like Fedwire and the power grid may have been established with good intentions, they have made the foundations of our society more fragile by making us dependent on institutions that act as single points of failure.
Now think about yet-to-be-implemented ideas like “lethal autonomous weapons systems”—militarized robots that kill people without direct human input. The existence of such systems would imply an incredibly dangerous search problem: “Is there a way to hack into these systems, activate them, and select their targets?” If you weren’t persuaded by my previous virtue-ethical argument against lethal autonomous weapons systems, maybe this consequentialist argument will persuade you!
Thesis 3
Christian ethics calls for us to reject the “technocratic paradigm” of simplification and centralization in favor of subsidiarity.
I began this essay with a quote from Exodus 20:4–6, which contains the image of God “punishing children for the iniqiuity of parents to the third and fourth generation of those who reject me, but showing steadfast love to the thousandth generation of those who love me and keep my commandments.” You might be puzzled as to why I chose to open my essay with such a strange, ominious, and maybe even repellent passsage.
What I mean to say is that the “technocratic paradigm” did not suddenly spring into existence a few years ago with the invention of generative AI. Rather, this is an old, old paradigm that has been handed down to us and reproduced over the course of generations, as illustrated by Pope Leo XIV’s interpretation of the story of the Tower of Babel in his recent encyclical on human dignity and AI, Magnifica humanitas:
Fearing being scattered across the earth, [the builders of the Tower of Babel] sought to guarantee stability and power for themselves, and above all to “make a name” for themselves. It was an impressive feat: a single language, a single technology, a single direction. However, the project concealed a profound danger. It is a project conceived without reference to God, supported by a uniformity that elimited diversity and that chose homogenization over communion. When a city is built on pride and the claim to self-sufficiency, communication breaks down, languages are confused and people no longer understand each other. The result is not unity, but dispersion. Babel thus reveals the limits of any effort that, however grandiose, arises from self-affirmation, sacrifices human dignity for efficiency and aspires to reach heaven without God’s blessing.3
We are the children being punished “for the iniquity of parents to the third and the fourth generation.” Historical processes like the Industrial Revolution and the emergence of the administrative nation-state as the basis of the political order have cemented the dominant role of the technocratic paradigm in human society. The recent development of powerful AI models that will become increasingly capable of solving dangerous search problems calls us to reckon with the structural weaknesses and flaws of our technocratically-ordered world. We could attempt to paper over the issues, improving cybersecurity to make our systems harder to attack, but that would just lead to a neverending arms race between cyberattackers and defenders.
There is a better way, and that way is subsidiarity. Subsidiarity is a concept that comes out of the social teaching of the Catholic Church. In this political model, “the role of individuals, families, local communities and intermediary organizations should not be supplanted by higher-level authorities,” but “higher-level institutions must recognize, protect, and promote the freedom and creativity of lower-level entities, coordinating their contributions so that they can cooperate effectively for the common good.”4
What if, for example, we increased the number of microgrids which can operate independently from centralized power grids? Then, it would be much more difficult for a single cyberattack to cut power to huge swaths of the population. Subsidiarity would thus restore much-needed complexity and resilience to our natural and social environments. In fact, if political and economic power was distributed throughout our society in accordance with subsidiarity, we might never have seen the level of capital concentration that was required to train large language models to begin with. As Pope Leo urges, we need technological transformations to “not be imposed from above in an opaque and unilateral manner, but instead by directed toward the common good with transparency, accountability, and meaningful forms of participation.”5
We would do well to ponder these words Christ spoke in the Gospel of Matthew:
“You know that the rulers of the gentiles lord it over them, and their great ones are tyrants over them. It will not be so among you, but whoever wishes to be great among you must be your servant, and whoever wishes to be first among you must be your slave, just as the Son of Man came not to be served but to serve and to give his life a ransom for many.”
James C. Scott, Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed (Yale University Press, 2020), 4–5 https://doi.org/10.12987/9780300252989.
Pope Francis, Laudato si’, May 24, 2015, para. 106 https://www.vatican.va/content/francesco/en/encyclicals/documents/papa-francesco_20150524_enciclica-laudato-si.html.
Pope Leo XIV, Magnifica humanitas, May 15, 2026, para. 7, https://www.vatican.va/content/leo-xiv/en/encyclicals/documents/20260515-magnifica-humanitas.html.
Magnifica humanitas, para. 68.
Magnifica humanitas, para. 71.