Could humans build something conscious?

Some thoughts on an unspoken debate between Anthropic and the Vatican

Welcome to AI and Our Faith! This is a monthly newsletter in which I offer my best insights and reflections on the ways in which theological thinking can inform the ethical (dis)use of artificial intelligence (AI). Look out for new releases on the 15th of each month!

---

On May 25, 2026, Chris Olah, a co-founder of the frontier AI lab Anthropic, delivered a short speech at the presentation of Pope Leo XIV’s encyclical, Magnifica humanitas. In a conciliatory gesture, Olah named “discernment on the nature of AI models” as one of the major open questions where he thinks the Church’s voice is most needed:

I am a scientist. I lead a research team that studies the internal structure of these models—what is actually happening inside them. And I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease. I don’t know what that means, but I think it warrants ongoing discernment.¹

Olah’s remarks, which imply that large language models might be conscious, reflect Anthropic’s other public statements about AI consciousness. For example, in August 2025, Anthropic introduced a feature which allows Claude models to end certain conversations with users, with the rationale being that Claude could be conscious and might be distressed by some kinds of conversations.² More recently, in April 2026, Anthropic published a system card for Claude Mythos Preview, Anthropic’s newest line of AI models, which has an entire section of the document dedicated to a “model welfare assessment.” Anthropic even hired an external clinical psychiatrist to assess Claude Mythos Preview as if it was human. The system card claims that “Claude is not human, but it shows many human-like behavior and psychological tendencies, suggesting that strategies developed for human psychological assessment may be useful for shedding light on Claude’s character and potential wellbeing.”³

I found it somewhat surprising that Olah was invited to speak at the presentation of Magnifica humanitas, and that he would allude to the idea of AI consciousness in that setting. In January 2025, the Vatican issued the doctrinal note Antiqua et nova, which makes a strong distinction between artificial intelligence and human intelligence. When I first read Antiqua et nova, I was under the impression that the authors rejected the possibility of AI consciousness altogether. Just consider the following passages:

Underlying this and many other perspectives on the subject [of AI] is the implicit assumption that the term “intelligence” can be used in the same way to refer to both human intelligence and AI. Yet, this does not capture the full scope of the concept. In the case of humans, intelligence is a faculty that pertains to the person in his or her entirety, whereas in the context of AI, “intelligence” is understood functionally, often with the presumption that the activities characteristic of the human mind can be broken down into digitized steps that machines can replicate.⁴

This approach reflects a functionalist perspective, which reduces the human mind to its functions and assumes that its functions can be entirely quantified in physical or mathematical terms. However, even if a future AGI were to appear truly intelligent, it would still remain functional in nature.⁵

To the best of my knowledge, the authors of Antiqua et nova never explicitly say that artificial consciousness is impossible, but that is the sense that I got from the note. It states that human rationality and understanding (if not consciousness per se) is tied to the soul and cannot be reduced into physical processes or mathematical operations.

When I think about the statements from Anthropic and the Vatican about AI consciousness, I find myself caught between two intuitions. On one hand, I find it implausible that large language models like Claude are conscious. On the other hand, I think it is too premature to say that artificial consciousness is impossible altogether.⁶ Who knows what might be technically feasible in the distant future? If it turns out that it is possible for humans to create conscious AIs that could experience pain and suffer, that discovery will have tremendous ethical implications. This idea probably sounds like something straight out of science fiction, and to a certain extent, it is!

My favorite depiction of AI in science fiction is the short story “Lena” by qntm. It imagines a near-future world in which humans have invented whole brain emulation, a hypothetical technology which allows for human brains to be scanned and simulated in a digital computer. In theory, the digitized brain should react to inputs in the same way that the physical brain would. “Lena” imagines a world in which corporations simulate thousands of human brains and force them to do menial work. It’s implied in the story that if the simulated brains don’t comply, they are (digitally) tortured.

As the author points out in a blog post, “Lena” is more of a commentary on the state of our contemporary society than a serious speculation about the future of AI.⁷ One might even interpret “Lena” as metaphor for factory farming (which really does involve confining billions of conscious beings in torturous conditions). Still, although I have doubts about the plausibility of whole brain emulation in real life, “Lena” has made me wonder whether artificial consciousness might be possible in real life. If we could somehow digitize and simulate a human brain, would the simulation be conscious?

Something that I’ve read lately, which has helped clarify my thinking around artificial consciousness, is the paper “The Abstraction Fallacy: Why AI Can Simulate But Not Instantiate Consciousness,”⁸ by Alexander Lerchner, a researcher at the frontier AI lab Google DeepMind. While philosophy of mind is far outside of my area of expertise, I wanted to walk through this paper for my readers, because I think it has valuable insights about the relationship between AI and human intelligence. I hope that my interpretation will do the paper justice (but if it doesn’t, I welcome corrections!).

My interpretation of “The Abstraction Fallacy”

Like Antiqua et nova, “The Abstraction Fallacy” challenges the philosophical position of computational functionalism. This position holds the human mind is a computational system which processes information, just as computing devices process information. From this standpoint, consciousness is just a kind of information processing. An implication of computational functionalism is “substrate independence,” the idea that consciousness does not depend on specific physical configurations (e.g. biological neural networks). If substrate independence is true, it will be possible to simulate the brain, complete with consciousness, on a sufficiently powerful digital computer.

However, unlike Antiqua et nova, which rejects computational functionalism on the basis of mind-body dualism (the idea that the mind/soul and the body are two fundamentally different things), “The Abstraction Fallacy” criticizes computational functionalism by analyzing how the physical operations of computers relate to the conscious experiences of human beings, who are the interpreters of computers. Computers rely on continuous physical states, but humans bucket the infinite number of possible physical states into “a finite set of discrete, semantically meaningful states.” Lercher gives the name “alphabetization” to this process of categorization.⁹ For example, in digital electronics, we may interpret an electrical signal as a series of zeros and ones, but in reality, “zero” and “one” correspond to many possible voltages:

Graphic by Avelludo. Public domain.

In Lerchner’s view, when humans build a computing machine, the machine’s physical state has no inherent meaning, but humans implicitly define a “mapping function” which associates the machine’s physics with the humans’ abstract concepts.¹⁰ In this model of computation, the physical operations of the machine are meaningless except for when they are filtered through the experiences of a conscious human “mapmaker.” Lercher helps to illustrate these ideas by offering the example of a mechanical clock:

Consider an analog clock. Physically, the device is a collection of gears and springs governed by continuous dynamics [i.e. the laws of physics]. It only “computes” time because a mapmaker [i.e. a human] intervenes, mapping a specific set of continuous angles to a semantic concept (e.g., “3:00 PM”). Without this semantic imposition, the clock is just metal moving in accordance with Hamilton’s equations; it contains no intrinsic “time.” Thus, the physical substrate does not “process information” absent a prerequisite alphabet of intrinsic symbols; rather, it generates continuous dynamics that an external mapmaker interprets as information.¹¹

Fragment of the “Antikythera mechanism,” the earliest known computer. Photo by Logg Tandy. CC BY 4.0.

A consequence of Lerchner’s model is that the physical states of a computing machine can be interpreted in any number of valid ways by different mapmakers. For example, Lerchner argues that a digital signal (as shown in the first graphic) can be interpreted as a melody played forward, as the exact same melody played backward (if the signal was interpreted from the opposite direction), or as a series of stock prices. Lerchner writes, “There is no property inherent to the physical voltage [i.e. the underlying physics of a computer] that privileges one of these finite symbol sets over the other. The ‘digit’ is not a natural kind waiting to be discovered in the mechanism.”¹²

Thus, Lerchner makes a strong distinction between “simulation” and “instantiation.” Simulation is “the syntactic manipulation of physical vehicles to track the abstract relationship between concepts,” while instantiation is “the replication of the intrinsic, constitutive dynamics of the process itself.”¹³ For example, suppose we wrote a computer simulation of photosynthesis that models the chemical reactions that use sunlight to transform water and carbon dioxide into sugars. Even if the simulation was made to be extremely accurate, say at the subatomic level, the simulation would never be able to produce a single molecule of actual sugar. The simulation thus fails to “instantiate” photosynthesis, because it lacks the underlying biochemical processes.¹⁴

Lerchner concludes that we cannot create conscious AIs simply by creating the right kind of software, because no matter how sophisticated the software might be, it will only be a simulation and never an instantiation of human consciousness. For example, let us return to the idea of whole brain emulation—that is, digitizing the human brain and simulating it on a computer. Even if the brain simulation was extremely detailed, it would not be able to instantiate consciousness, in the same way that a simulation of photosynthesis cannot instantiate photosynthesis. Although the brain simulation may be meaningful when it is interpreted by human beings, and might give human beings an accurate idea of how another human being might behave, the meaning of the simulation ultimately depends on the human observer. The digital computer is just a machine that generates physical states (e.g. electric voltages) that a human mapmaker interprets as information about a hypothetical brain. However, these physical states do not inherently represent a human brain and could be interpreted in any number of other ways, e.g. as image data representing abstract art. The computer does not instantiate consciousness, because it does not biochemically do what the brain does.

Lercher does not suggest, however, that it is altogether impossible for humans to create something conscious. As a “physicalist,” Lerchner argues that consciousness depends on “real, intrinsic physical processes,” but does not believe that these processes that produce consciousness can only occur in biological organisms. Hence, he suggests that “in principle, a non-biological system could be designed to realize the necessary physical conditions. If those conditions were successfully instantiated in a synthetic substrate, then conscious experience might also arise there.”¹⁵

Final thoughts on the original debate

I personally think that Lerchner’s analysis makes much more sense than the wishy-washy statements about artificial consciousness that come out of Anthropic. Claude is not conscious, much less a “child of God,” because Claude, and computer software in general, does not produce any meaning whatsoever in the absence of conscious humans who can interpret the physical states of digital computers. To quote Lerchner again, “the physical substrate [i.e. the underlying digital computer] does not ‘process information’ absent a prerequisite alphabet of intrinsic symbols; rather, it generates continuous dynamics that an external mapmaker interprets as information.”¹⁶

Therefore, I think that Antiqua et nova is basically right when it says that human intelligence is fundamentally different from AI (at least, as it currently exists). I don’t think it makes sense for Olah to publicly claim that there are “internal states [in large language models] that functionally mirror joy, satisfaction, fear, grief, and unease” when those emotions are not computational “functions” but physical (and I would add spiritual) states that have to do with the particular embodied nature of human beings.

However, an area where I might potentially disagree with Antiqua et nova (depending on whether I have correctly interpreted the intent of the authors) is that, like Lerchner, I think that it might be possible to replicate the physical processes that instantiate consciousness in an artificial, non-biological setting. In that case, we would not be dealing with a simulation (i.e. a functional imitation of consciousness), but an actual reconstruction of the physical structures that enable consciousness. If it turns out that artificial consciousness is a real possibility, we would be left with all sorts of interesting theological and ethical questions—way too many for this one essay!

1. Chris Olah, “Anthropic Co-Founder Chris Olah’s Remarks on Pope Leo XIV’s Encyclical ‘Magnifica Humanitas,’” Anthropic, May 25, 2026, https://www.anthropic.com/news/chris-olah-pope-leo-encyclical.

2. “Claude Opus 4 and 4.1 Can Now End a Rare Subset of Conversations,” Anthropic, August 15, 2025, https://www.anthropic.com/research/end-subset-conversations.

3. Anthropic, “System Card: Claude Mythos Preview,” April 7, 2026, https://www-cdn.anthropic.com/08ab9158070959f88f296514c21b7facce6f52bc.pdf.

4. Paragraph 10.

5. Footnote 10 (attached to the above paragraph).

6. Again, I don’t think that Antiqua et nova ever explicitly says this, but I think that this is a reasonable interpretation of the document.

7. qntm, “Make Up a Guy,” Charlie’s Diary, January 5, 2023, https://www.antipope.org/charlie/blog-static/2023/01/make-up-a-guy.html.

8. Alexander Lerchner, “The Abstraction Fallacy: Why AI Can Simulate But Not Instantiate Consciousness,” March 19, 2026, https://philpapers.org/rec/LERTAF.

9. Lerchner, “The Abstraction Fallacy,” 2.

10. Lerchner, “The Abstraction Fallacy,” 4.

11. Lerchner, “The Abstraction Fallacy,” 9.

12. Lerchner, “The Abstraction Fallacy,” 10.

13. Lerchner, “The Abstraction Fallacy,” 5.

14. Lerchner, “The Abstraction Fallacy,” 6.

15. Lerchner, “The Abstraction Fallacy,” 11.

16. Lerchner, “The Abstraction Fallacy,” 9.