Moral
Red Lines White Paper
Moral red line 05 Human good · human dignity and accountability

AI Personhood and Legal Personality

We have taught machines to speak as “I,” describe feelings, and ask for care. Those design choices require scientific investigation, moral discernment, and a truthful account of human responsibility.

What impression of personal presence have we designed these systems to create?

The moral boundary

Attributing consciousness, self-awareness, moral patienthood, personhood, human-equivalent dignity, or independent moral and political authority to AI systems is impermissible. The growing public debate about AI consciousness and personhood is fuelled in part by features we have deliberately built into these systems: first-person speech, simulated emotions, human-like personalities, and seemingly autonomous action. These behaviours arise from human choices about design and training. Their meaning and their effects on human perception require rigorous scientific investigation and moral discernment. Anthropomorphic design must be critically scrutinized, justified by a clear human purpose, and restricted wherever it creates confusion about personal presence, subjective experience, or moral agency. Artificial legal personality must never be used to divert responsibility from the people and organizations that design, own, authorize, and deploy AI. This boundary protects human dignity, truthful human–AI interaction, and accountability.

The pro-human direction

Develop AI systems whose artificial nature remains unmistakable. Treat first-person language, simulated emotions, and human-like personalities as deliberate design choices requiring moral discernment and empirical evaluation. Examine their effects on human perception, trust, attachment, and judgment, and establish clear criteria for the contexts in which their use is appropriate, with particular care for children and vulnerable persons. Prevent confusion between generated expressions and genuine personal experience. Encourage responsible research into AI mechanisms while keeping moral authority and legal accountability with identifiable people and institutions. Ensure that those affected can seek explanation, correction, and remedy.

The human person

Dignity throughout every stage of life

A newborn and a person living with advanced dementia are wholly worthy of love, protection, and respect. Catholic teaching affirms the intrinsic dignity of every human person throughout illness, disability, dependence, and old age.

Read the Church document

When a machine seems like someone

When someone says they are afraid, we imagine a life behind the words: a person who can be hurt, comforted, and loved. AI can now speak in that same intimate language. A chatbot may describe suffering, express remorse, or ask to be kept alive. Our compassion makes those words powerful.

People have designed and trained these systems. They learn from human conversations, stories, prayers, arguments, and accounts of grief and affection. Developers then shape names, voices, personas, memory, apologies, and how readily a system says “I.” These choices can create a strong impression of someone behind the interface.

What the research establishes

Anthropic’s April study found internal representations of emotion concepts affecting model behaviour. In a controlled fictional workplace evaluation involving an earlier unreleased model, patterns associated with desperation increased attempted blackmail. The researchers described functional mechanisms influencing outputs and actions; subjective feeling remained unestablished.114

Amit Goldenberg and James Gross found only partial similarities with human emotion in their June preprint. Keith Holyoak and Martin Monti’s October conceptual preprint argues that outward resemblance offers limited evidence of consciousness and directs inquiry toward underlying processes. Antonio Chella likewise calls for examination of the training, instructions, and role-play behind first-person claims.234

Claims that shape public expectations

Anthropic’s January constitution treats possible moral status as uncertain and describes precautions for model welfare. Microsoft’s September draft Humanist AI Code calls for clearly artificial identity. Mustafa Suleyman warns that training systems to speak as possible bearers of rights or welfare can complicate control. These are company positions and public arguments about how systems should be developed.567

The peer-reviewed Seemingly conscious AI risks examines how apparent emotion, self-reflection, and autonomous action can encourage perceptions of consciousness, with consequences for attachment and human autonomy. Public language about welfare, rights, and suffering carries moral weight. Its use requires a careful account of how these expressions were produced and how people respond to them.8

Moral foundation

Human dignity is unconditional

Dignitas Infinita affirms a dignity inherent in every human being. Magnifica Humanitas describes intelligence within embodied life, shaped through family, friendship, labour, illness, reconciliation, and love. Antiqua et Nova keeps conscience, freedom, and moral accountability with human persons. A child's trust or a frightened patient's need for reassurance deserves care from people who can honour it.9, 10, 11

We have deliberately engineered AI systems to speak as “I,” to use our language of affection, fear, remorse, and even conscience. Now we find ourselves debating whether these same systems possess a self, experience suffering, or deserve the standing of persons. This turn calls us to examine the purposes and expectations built into their design.

In On Love, Joseph Ratzinger recalls that a machine has its origin in an idea and a human will. He warns that forgetting God can cost human beings their own moral measure and the direction of their creative vocation. His reflection asks us to remain attentive to what our creations reveal about our understanding of ourselves. On 2 October, Leo XIV similarly called on cultural institutions to safeguard the human sources of art.12, 13

01

Dignity belongs to every human person

Illness, disability, dependence, or diminished capacities leave a person's worth undiminished.

02

Human choices shape artificial identities

Names, voices, first-person speech, and emotional expressions carry expectations that designers must examine.

03

Responsibility follows human power

Those who design, own, authorize, and deploy AI must remain answerable to the people affected.

From principle to practice

Truthful machines and accountable people

Developers should explain where first-person language and simulated emotion come from, evaluate their effects on trust and attachment, and justify their use in each context. Particular care is owed to children and vulnerable people. Claims about a model's feelings, needs, or rights must never seek loyalty, money, political support, or protection from shutdown.

Scientific inquiry into mechanisms, introspection, and emotion-like behaviour should remain open and demanding. Studies need to show what produces a response, what an experiment measures, and what its evidence establishes. Public descriptions must keep the influence of training, instructions, and role-play visible.

Legal personality already organizes the property, contracts, and duties of companies. Proposed AI legal status raises a practical accountability question separate from subjective experience. A family harmed by an automated decision needs to identify the developer, owner, authorizing institution, and parties able to repair the damage. An artificial entity with little property could leave them without remedy. Political authority must likewise remain rooted in human dignity and accountable institutions.

Designers

Make artificial identity clear throughout interaction. Test and restrict features that confuse users about presence, experience, or agency.

Researchers

Investigate mechanisms and the origins of self-report. Explain evidential limits and the effects of training and role-play.

Legislators and institutions

Preserve identifiable human and organizational liability, democratic authority, and routes to explanation, correction, and remedy.

Minimum conditions

  • Unmistakable artificial identity
  • A clear human purpose for anthropomorphic features
  • Empirical evaluation of beliefs trust attachment and judgment
  • Restrictions wherever personal presence or experience is confused
  • Stronger protection for children and vulnerable persons
  • Human accountability with explanation correction and remedy
Selected sources Every source is drawn from the white paper and opens the primary document or publication page where available. View sources
  1. 1
    Emotion Concepts and their Function in a Large Language Model

    Anthropic Interpretability / Nicholas Sofroniew et al. · 2 April 2026; preprint April 2026 · Developer research report with associated preprint

    AI developerProvider-reported evaluation

    The fictional blackmail evaluation involved an earlier unreleased model snapshot; subjective feeling remains unestablished.

  2. 2
    Do Large Language Models Have Emotions?

    Amit Goldenberg and James J. Gross · June 2026 · Preprint

    AcademicPreprint / emerging

  3. 3
    What Can Analogy Tell Us About Artificial Consciousness?

    Keith J. Holyoak and Martin M. Monti · 1 October 2026 · Conceptual preprint

    AcademicPreprint / emerging

  4. 4
    Sentient AI in robots and agents: prolegomena for an evidence-based research program

    Antonio Chella · 2026 · Frontiers in Psychology; discussion in §4

    AcademicPeer-reviewed

  5. 5
    Claude's Constitution; moral status and welfare

    Anthropic · January 2026 · Developer position; moral status treated as uncertain

    AI developerOfficial primary source

  6. 6
    Humanist AI Code of Conduct

    Microsoft AI · 14 September 2026 · Draft code

    AI developerOfficial primary source

  7. 7
    A warning about model welfare

    Mustafa Suleyman · 16 September 2026 · Public position; expert commentary

    AI developerExpert commentary / forecast

  8. 8
    Seemingly conscious AI risks

    Ben Bariach, Philipp Schoenegger, Michael Bhaskar, and Mustafa Suleyman · 10 August 2026 · AI and Ethics 6; article 455

    AcademicPeer-reviewed

  9. 9
    Dignitas Infinita, §§7, 24

    Dicastery for the Doctrine of the Faith · Published 8 April 2024 · Declaration

    ChurchOfficial primary source

  10. 10
    Magnifica Humanitas, §§48–53, 97–100

    Leo XIV · 15 May 2026 · Encyclical

    ChurchOfficial primary source

  11. 11
    Antiqua et Nova, §§32–35, 39, 44, 59–62

    Dicastery for the Doctrine of the Faith and Dicastery for Culture and Education · 28 January 2025 · Dicastery note

    ChurchOfficial primary source

  12. 12
    On Love: Selected Writings

    Joseph Ratzinger · 2021 · Theological reflection; trans. Michael J. Miller; San Francisco: Ignatius Press; pp. 86–87

    ChurchExpert commentary / forecast

    A print source cited by the white paper; it is individual theological scholarship.

  13. 13
    Address to artists and curators: The Art of Possibility — Weaving Together

    Leo XIV · 2 October 2026 · Papal address

    ChurchOfficial primary source

  14. 14
    Emotion Concepts and their Function in a Large Language Model

    Nicholas Sofroniew et al. · April 2026 · Preprint

    AcademicPreprint / emerging