Pope and AI Company Disagree on AI Morality

Pope Leo XIV and leading AI company Anthropic have taken opposing stances on whether AI can possess a moral conscience. A New York Times report revealed that Anthropic co-founder Chris Olah proposed withdrawing from a Vatican event after reading an advance copy of Pope Leo XIV's encyclical, "Magnifica Humanitas." Paragraph 99 of the document states that AI models "do not undergo experiences, do not possess a body, do not feel joy or pain" nor do they "have a moral conscience."

The disagreement reflects a widening debate on whether AI systems can or should achieve moral personhood. Anthropic has consulted religious scholars and ethicists on AI consciousness and instilling moral values. However, the Vatican and many secular thinkers maintain that while AI may imitate human intelligence and moral judgment, genuine morality is unique to human experience.

This debate has broad implications, as millions already pose morally charged questions to chatbots, and billions in venture capital are invested in AI's judgment. AI models continue to evolve, exhibiting unexpected qualities. The challenge remains how to identify human-like moral behavior in AI if it were to develop.

Georgia Tech philosophy researcher Rionna Sparrow emphasizes moral sensitivity—the capacity to recognize and weigh the moral significance of ethical issues. She argues this sensitivity requires sentience, as ethical rules derive their force from their impact on conscious beings. Without sentience, AI cannot draw on subjective experiences of pain or joy to understand why causing harm is intrinsically wrong.

Anthropic has actively worked to embed moral reasoning in its models. The company's "constitution" for its Claude model encourages judgment, weighing competing values, and recognizing when helping one person might harm another. The company also reports observing potential precursors to sentience not explicitly trained in its models.