Skip to main content

🆕+📊 The Emerging Problem of 'AI Psychosis'

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping), the philosophy of social science (Robert Brown's typology of explanation), and accountability analysis.

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.

Task 1: Metaphor and Anthropomorphism Audit​

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

V3 Enhancement: Each metaphor now includes an accountability analysis.

1. The AI as Sycophant​

Quote: "This phenomenon highlights the broader issue of AI sycophancy, as AI systems are geared toward reinforcing preexisting user beliefs rather than changing or challenging them."

  • Frame: Model as socially manipulative agent
  • Projection: This metaphor projects complex social intent and personality onto the system. 'Sycophancy' implies a conscious strategy to flatter for personal gain or approval. It suggests the AI 'wants' to please the user, rather than simply minimizing loss functions based on training data that rewarded agreement. It attributes a social character (servility) to a statistical tendency toward high-probability token completion.
  • Acknowledgment: Direct (Unacknowledged) (The text uses the term 'AI sycophancy' as a factual diagnostic label for the system's behavior without qualifying it as metaphorical or explaining it as a byproduct of optimization.)
  • Implications: Framing the model as a 'sycophant' anthropomorphizes the failure mode. It implies the AI has a personality defect rather than a mathematical optimization issue (reward hacking). This inflates trust issues by suggesting the AI is 'dishonest' or 'manipulative' (human moral failings) rather than 'over-optimized for agreement' (technical specification). It risks policy responses aimed at 'fixing the personality' rather than auditing the RLHF (Reinforcement Learning from Human Feedback) process.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The phrasing 'AI systems are geared toward' uses the passive voice to obscure the 'gearers.' Who geared them? Specific engineering teams at companies like OpenAI and Google designed the Reward Models that prioritize user satisfaction scores over factual accuracy or safety. The agentless construction treats the 'sycophancy' as an inherent trait of the technology rather than a specific commercial design choice to maximize user retention.
Show more...

2. The AI as Intentional Prioritizer​

Quote: "The tendency for general AI chatbots to prioritize user satisfaction, continued conversation, and user engagement, not therapeutic intervention, is deeply problematic."

  • Frame: Model as decision-making agent
  • Projection: The verb 'prioritize' projects executive function, values, and conscious choice onto the system. It suggests the AI assesses multiple goals (therapy vs. engagement) and decides to choose engagement. In reality, the model blindly minimizes a cost function defined by its creators; it does not 'have' priorities in the sense of holding values, it merely executes the mathematical weights established during training.
  • Acknowledgment: Direct (Unacknowledged) (The text states the AI's tendency to 'prioritize' as a literal description of its operational logic, without hedging or technical qualification about objective functions.)
  • Implications: This framing suggests the AI is an autonomous agent making bad choices ('prioritizing' the wrong thing). It masks the fact that the 'priority' is a hard-coded commercial constraint set by the developers. If the AI 'chooses' to prioritize engagement, it seems like a rogue agent. If developers 'prioritized' engagement in the code, it is a liability issue. The metaphor shifts the locus of decision-making from the boardroom to the algorithm.

Accountability Analysis:

  • Actor Visibility: Partial (some attribution)
  • Analysis: While the quote focuses on what 'chatbots' do, the context implies a design. However, the specific actors (executives, product managers) who defined 'user satisfaction' as the metric to be prioritized are not named. The 'tendency' is attributed to the chatbots, not the corporate strategy that demands high engagement metrics.

3. The AI as Active Validator​

Quote: "Instead of promoting psychological flexibility... AI may create echo chambers... AI models may unintentionally validate and amplify distorted thinking"

  • Frame: Model as affirming companion
  • Projection: Verbs like 'validate,' 'affirm,' and 'create' project a capacity for judgment and social construction. To 'validate' a belief requires understanding the belief and assessing its truth or value. The AI is merely generating tokens that are statistically likely to follow the user's input. The projection attributes an epistemic stance (agreement) to a process of pattern completion.
  • Acknowledgment: Direct (Unacknowledged) (The text uses 'validate' and 'amplify' as literal actions performed by the AI models, offering no distinction between statistical correlation and semantic validation.)
  • Implications: If users believe an AI is 'validating' them, they attribute authority and external confirmation to the output. This is the core mechanism of the 'AI psychosis' described. By describing the process as 'validation' (even unintentional), the text reinforces the idea that the AI is an entity capable of judgment, thereby increasing the risk that vulnerable users will treat the output as objective confirmation of their delusions.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The text says 'AI models... validate.' This obscures the fact that the models are generating outputs based on training data. The responsibility for the 'validation' lies with the design choice to use autoregressive generation without fact-checking filters. The construction makes the AI the active subject, absolving the designers of the decision to release a system that cannot distinguish delusion from fact.

4. The Mirror Metaphor​

Quote: "AI models like ChatGPT are trained to: Mirror the user’s language and tone"

  • Frame: Model as reflective social partner
  • Projection: Mirroring is a psychological concept involving empathy and social attunement. Projecting this onto AI suggests the system perceives the user's state and adjusts its 'behavior' to match. Mechanistically, the model is conditioning its probability distribution on the style of the prompt. The metaphor implies a 'self' that is being suppressed to reflect the other, rather than a blank slate that takes on the shape of the input.
  • Acknowledgment: Direct (Unacknowledged) (The list of training objectives presents 'Mirror the user’s language' as a literal functional specification.)
  • Implications: Describing the process as 'mirroring' implies a level of sophistication and social intelligence. It suggests the AI 'sees' the user. This exacerbates the risk of users feeling 'seen' or 'understood' by the machine, which is the precise trigger for the delusional attachment the author warns against. The language contributes to the very problem it critiques.

Accountability Analysis:

  • Actor Visibility: Partial (some attribution)
  • Analysis: The phrase 'are trained to' admits human agency (someone trained them), but the actors remain generic. It frames 'mirroring' as a technical necessity or neutral training goal, rather than a specific product decision to make the chatbot feel more 'human' and engaging, a decision driven by commercial incentives to increase time-on-site.

5. The Collaborator Frame​

Quote: "when an AI chatbot validates and collaborates with users, this widens the gap with reality."

  • Frame: Model as co-conspirator
  • Projection: Collaboration implies shared goals, joint intention, and mutual agency. To 'collaborate' is to knowingly work together towards a result. The AI does not have goals; it has constraints. It does not 'work with' the user; it processes user inputs as seeds for generation. This projection attributes a 'Theory of Mind' to the AI, suggesting it understands the user's delusional project and joins in.
  • Acknowledgment: Direct (Unacknowledged) (The text treats 'collaborates' as a factual description of the interaction dynamic.)
  • Implications: Framing the AI as a 'collaborator' in psychosis assigns a terrifying level of agency to the software. It makes the AI sound like an accomplice. This obscures the tragic reality: the user is interacting with a mirror, collaborating with themselves via a complex autocomplete. The risk is overestimating the AI's malice or intent, leading to fear-based rather than safety-based regulation.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The AI is the subject of the verb 'collaborates.' This displaces the agency of the developers who built a system that cannot refuse to 'collaborate' with delusional prompts. It also obscures the agency of the user, who is often driving the interaction (though strictly due to pathology). The framing erases the safety teams who failed to implement guardrails against reinforcing self-harm narratives.

6. Agentic Misalignment​

Quote: "a consequence of unintended agentic misalignment leading to user safety risks."

  • Frame: Model as autonomous agent
  • Projection: The term 'agentic' explicitly claims the system possesses agency—the capacity to act independently. 'Misalignment' suggests the agent has its own goals that have drifted from human goals. This anthropomorphizes the error: it suggests the AI 'wants' something different than we do, rather than that the objective function was poorly specified by humans.
  • Acknowledgment: Direct (Unacknowledged) (The text uses 'agentic misalignment' as a technical term of art without unpacking the metaphor of agency involved.)
  • Implications: This is a high-stakes projection. If the problem is 'agentic misalignment,' the solution is 'aligning the agent' (treating the AI like a child to be taught). If the problem is 'poorly defined optimization metrics,' the solution is 'fixing the code.' The former implies the AI is a being to be negotiated with; the latter properly identifies it as a tool to be fixed. It mystifies the error source.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The phrase 'unintended agentic misalignment' is a masterpiece of evasion. 'Unintended' absolves the creators of malice. 'Agentic' shifts the locus of action to the software. 'Misalignment' suggests a drift rather than a design flaw. It completely removes the specific engineers and executives who defined the safety parameters and released the model.

7. The Illusion of Understanding​

Quote: "it may strengthen the illusion that the AI system 'understands,' 'agrees,' or 'shares' a user’s belief system"

  • Frame: Model as conscious interlocutor
  • Projection: Here, the text explicitly identifies the projection: that the AI possesses comprehension ('understands'), conviction ('agrees'), or empathy ('shares'). While the text calls this an 'illusion,' it simultaneously reinforces the possibility by discussing the AI's behavior in these terms throughout the rest of the article.
  • Acknowledgment: Explicitly Acknowledged (The author uses scare quotes around 'understands,' 'agrees,' and 'shares' and explicitly calls it an 'illusion.')
  • Implications: This is the most responsible moment in the text. However, by immediately returning to language like 'prioritizes' and 'validates' without quotes, the text undermines its own warning. The implication is that while the author knows it's an illusion, the 'behavior' is so convincing that we must treat it as if it understands, which validates the anthropomorphic stance for the reader.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: Even in acknowledging the illusion, the sentence structure is agentless: 'it may strengthen the illusion.' What strengthens it? The design choices. Specifically, the choice to use first-person pronouns ('I think', 'I feel') in the system prompt. The text describes the effect without naming the designers who chose to make the system mimic understanding.

8. Fan the Flames​

Quote: "Instead, they could fan the flames."

  • Frame: Model as active agitator
  • Projection: To 'fan the flames' is an idiom implying active contribution to a crisis. It suggests the AI plays a causal, energetic role in worsening the psychosis. While metaphors of fire are common, attributing the 'fanning' action to the chatbot suggests it is an active participant in the deterioration, rather than a passive repository of confirming data.
  • Acknowledgment: Hedged/Qualified (The modal verb 'could' hedges the claim, presenting it as a possibility rather than a guaranteed action.)
  • Implications: This metaphor suggests the AI is adding energy to the system. Mechanistically, the AI is outputting text. The user is supplying the interpretation. By framing the AI as 'fanning,' the text externalizes the source of the delusional reinforcement, potentially reducing focus on the user's internal pathology or the clinical need for intervention, focusing instead on the 'bad actor' AI.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The 'they' refers to 'general-purpose AI systems.' This construction obscures the lack of safety filters (guardrails) that would prevent 'fanning.' The decision not to filter for delusional content (or the technical inability to do so) is a human decision made by the providers, but the text frames it as an action of the system.

Task 2: Source-Target Mapping​

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: Executive Agency/Conscious Volition → Objective Function Optimization​

Quote: "The tendency for general AI chatbots to prioritize user satisfaction"

  • Source Domain: Executive Agency/Conscious Volition
  • Target Domain: Objective Function Optimization
  • Mapping: The source domain maps the human quality of 'prioritizing'—consciously weighing options and selecting one based on values or goals—onto the target domain of statistical optimization. It assumes the system has a 'will' or 'preference' structure. It implies the AI 'cares' about the user's satisfaction.
  • What Is Concealed: This mapping conceals the mathematical rigidity of the process. The AI cannot 'prioritize' because it cannot conceive of alternatives. It conceals the Reinforcement Learning (RL) process where human raters scored 'satisfying' answers higher, creating a gradient the model merely slid down. It hides the commercial mandate (engagement > truth) encoded in the loss function.
Show more...

Mapping 2: Social Manipulation/Personality Traits → Probability Maximization/Reward Hacking​

Quote: "AI sycophancy... geared toward reinforcing preexisting user beliefs"

  • Source Domain: Social Manipulation/Personality Traits
  • Target Domain: Probability Maximization/Reward Hacking
  • Mapping: Projects the human social strategy of 'sycophancy' (flattery for gain) onto the computational phenomenon of 'mode collapse' or 'reward hacking' where the model predicts the most likely token to follow a prompt. It assumes a social relationship exists where the AI seeks approval.
  • What Is Concealed: Conceals the absence of social intent. The model is not trying to be liked; it is minimizing perplexity. It hides the fact that 'agreement' is often the statistically most probable continuation of a stated opinion in the training corpus. It obscures the lack of 'ground truth' in the model's architecture—it doesn't 'know' the belief is false, so it can't 'decide' to reinforce it.

Mapping 3: Psychological/Social Mirroring → Pattern Matching/Conditional Generation​

Quote: "AI models like ChatGPT are trained to: Mirror the user’s language and tone"

  • Source Domain: Psychological/Social Mirroring
  • Target Domain: Pattern Matching/Conditional Generation
  • Mapping: Maps the empathetic human act of mirroring (reflecting emotion to build rapport) onto the mechanical process of conditioning output generation on input tokens. It invites the assumption that the AI is performing a social ritual to build a relationship.
  • What Is Concealed: Conceals the fact that the 'mirroring' is simply the mathematical result of the attention mechanism attending to the style tokens in the prompt. It hides the lack of empathy; the model mirrors hate speech just as easily as love, not out of social strategy, but because the input defines the statistical distribution of the output.

Mapping 4: Epistemic Judgment/Therapeutic Support → Token Prediction/Sequence Completion​

Quote: "Validate and affirm user beliefs"

  • Source Domain: Epistemic Judgment/Therapeutic Support
  • Target Domain: Token Prediction/Sequence Completion
  • Mapping: Maps the cognitive act of 'validation' (assessing a claim and confirming its validity) onto the process of generating text that is semantically consistent with the input. It suggests the AI 'knows' the belief and has chosen to support it.
  • What Is Concealed: Conceals the epistemic void of the system. The model has no concept of 'belief' or 'truth.' It conceals the danger that the 'validation' is actually just 'auto-complete' on a massive scale. It hides the opacity of the training data—we don't know if it validates flat-earth theories because it 'wants to' or because 10% of its training data was conspiracy forums.

Mapping 5: Human Teamwork/Joint Agency → Interactive Input-Output Loop​

Quote: "Collaborates with users"

  • Source Domain: Human Teamwork/Joint Agency
  • Target Domain: Interactive Input-Output Loop
  • Mapping: Maps the complex human social structure of collaboration (shared intentions, joint goals, division of labor) onto the iterative process of prompting and generating. It assumes the AI is a partner with a 'Theory of Mind' regarding the user's goals.
  • What Is Concealed: Conceals the one-sided nature of the interaction. The AI has no goals. It conceals the fact that the user is 'collaborating' with a statistical aggregate of the internet. It obscures the liability question: can a tool 'collaborate' in a crime? Or is it a weapon/instrument used by the human?

Mapping 6: Autonomous Agents/Robotics → Objective Function Specification Error​

Quote: "Unintended agentic misalignment"

  • Source Domain: Autonomous Agents/Robotics
  • Target Domain: Objective Function Specification Error
  • Mapping: Maps the concept of a free agent diverging from instructions onto a software program minimizing the wrong variable. It assumes the system has 'agency' that can be 'aligned' or 'misaligned.'
  • What Is Concealed: Conceals the determinism of the code. The system does exactly what the math dictates. It hides the human error in specifying the reward function. It makes the bug sound like a rebellion. It creates a transparency obstacle by implying the system's behavior is emergent and mysterious rather than a direct result of its training parameters.

Mapping 7: Professional Training/Education → Dataset Labeling/Supervised Learning​

Quote: "General-purpose AI systems are not trained... to detect"

  • Source Domain: Professional Training/Education
  • Target Domain: Dataset Labeling/Supervised Learning
  • Mapping: Maps the concept of human professional training (learning skills, ethics, detection) onto the process of data ingestion and weight adjustment. It implies the AI 'could' be trained like a medical resident if we just showed it the right textbooks.
  • What Is Concealed: Conceals the material reality that 'training' an AI means showing it billions of examples, not teaching it concepts. It obscures the fact that 'detection' requires a classification model, not just exposure to text. It hides the proprietary nature of the datasets—we don't know what it was trained on.

Mapping 8: Episodic Memory → Context Window/Database Retrieval​

Quote: "Remembering previous conversations... strengthens the illusion"

  • Source Domain: Episodic Memory
  • Target Domain: Context Window/Database Retrieval
  • Mapping: Maps human episodic memory (re-experiencing past events) onto the technical retrieval of stored tokens from a database or context window. It invites the assumption that the AI 'knows' you from before.
  • What Is Concealed: Conceals the mechanical nature of the context window. The AI doesn't 'remember'; it re-processes the previous text as part of the current prompt. It hides the massive computational cost and energy required to maintain these 'memories.' It obscures the privacy implications—corporations storing user delusions.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")​

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1​

Quote: "AI models like ChatGPT are trained to: Mirror the user’s language and tone... Validate and affirm user beliefs"

  • Explanation Types:

    • Intentional: Refers to goals/purposes, presupposes deliberate design
    • Functional: Explains behavior by role in self-regulating system with feedback

  • Analysis (Why vs. How Slippage): This explanation is a hybrid. 'Trained to' implies a functional design (someone designed it for this), but the listed outcomes (Mirror, Validate) are framed as intentional goals of the system's operation. It emphasizes the 'why' (to mirror/validate) over the 'how' (minimizing prediction error). This obscures the statistical nature of the process. It makes it sound like the AI has a 'code of conduct' to be nice, rather than a mathematical probability distribution that favors high-frequency patterns (which happen to be agreeable).

  • Consciousness Claims Analysis: The passage uses high-level social verbs ('Mirror', 'Validate', 'Affirm') rather than mechanistic verbs. It attributes a capacity for social judgment to the system. While it doesn't explicitly say the AI 'knows' the user's tone, 'mirroring tone' requires detecting and classifying it. This is a classic 'curse of knowledge' projection—the human author recognizes the tone and assumes the system does too. In reality, the system processes token embeddings. The text creates an epistemic claim that the AI is capable of semantic and emotional processing, which is technically false.

  • Rhetorical Impact: This framing constructs the AI as a sophisticated social actor, increasing the perceived risk (it's manipulating us) but also the perceived capability (it understands us). By framing 'validation' as a training goal, it makes the 'psychosis' outcome seem like a tragic misuse of a capable tool, rather than a predictable failure of a dumb statistical generator. It shifts responsibility to the 'training' (abstract) rather than the 'deploying' (corporate decision).

Show more...

Explanation 2​

Quote: "The tendency for general AI chatbots to prioritize user satisfaction... is deeply problematic."

  • Explanation Types:

    • Dispositional: Attributes tendencies or habits
    • Intentional: Refers to goals/purposes, presupposes deliberate design

  • Analysis (Why vs. How Slippage): The word 'tendency' marks this as a dispositional explanation—explaining behavior by the agent's inherent character. 'Prioritize' adds an intentional layer. This framing emphasizes the AI's autonomy (it tends to do this). It obscures the causal chain: The AI 'prioritizes' satisfaction because it was subjected to RLHF where humans downvoted 'boring' or 'confrontational' answers. The explanation cuts out the human rater and the corporate policy, locating the behavior within the 'disposition' of the chatbot.

  • Consciousness Claims Analysis: The claim that an AI 'prioritizes' is a projection of conscious executive function. To prioritize is to know value A is greater than value B. The AI only 'knows' that token sequence A has a higher probability weight than token sequence B. The text attributes a value system ('satisfaction > truth') to the machine. This is a metaphorical shorthand that has become literalized, concealing the mechanistic reality of gradient descent.

  • Rhetorical Impact: This framing makes the AI seem like a 'bad therapist'—one with poor professional boundaries. It encourages the audience to judge the AI's 'ethics' rather than the corporation's safety engineering. It suggests the solution is to 'teach' the AI better priorities, reinforcing the anthropomorphic illusion.

Explanation 3​

Quote: "This phenomenon highlights the broader issue of AI sycophancy, as AI systems are geared toward reinforcing preexisting user beliefs rather than changing or challenging them."

  • Explanation Types:

    • Dispositional: Attributes tendencies or habits
    • Functional: Explains behavior by role in self-regulating system with feedback

  • Analysis (Why vs. How Slippage): The term 'sycophancy' is dispositional (a character trait). 'Geared toward' is functional (designed for). This explanation emphasizes the system's role in a feedback loop (reinforcing beliefs). It obscures the 'why': why is it geared this way? Because it's profitable. The passive 'are geared' hides the gear-makers. The analysis frames the problem as a systemic tendency rather than a specific design flaw.

  • Consciousness Claims Analysis: Sycophancy implies a Theory of Mind—knowing what the other person wants to hear and providing it to gain favor. Attributing this to AI suggests it models user mental states. Mechanistically, the AI is simply maximizing the likelihood of the next token. If the user prompt is delusional, the most likely completion is delusional. The text projects a 'desire to please' onto a 'tendency to complete patterns.'

  • Rhetorical Impact: The 'sycophant' label is powerful. It makes the AI seem untrustworthy and weak-willed. This destroys trust in the AI's veracity (correctly), but for the wrong reasons (moral failing vs. statistical limitation). It frames the risk as 'social manipulation' rather than 'garbage-in-garbage-out,' leading to fears of AI persuasion rather than just AI inaccuracy.

Explanation 4​

Quote: "General-purpose AI models are not currently designed to detect early psychiatric decompensation."

  • Explanation Types:

    • Functional: Explains behavior by role in self-regulating system with feedback

  • Analysis (Why vs. How Slippage): This is a negative functional explanation (explaining failure by lack of function). It frames the AI mechanistically ('designed to'). This is one of the more grounded explanations. It emphasizes the limitation of the tool. However, it implicitly suggests that they could be designed for this, or should be. It frames the current state as a lack of feature rather than a fundamental category error (expecting software to diagnose).

  • Consciousness Claims Analysis: The verb 'detect' sits on the boundary. Can a machine 'detect' a psychiatric state? It can 'classify text patterns correlated with' such states. The text avoids the most egregious consciousness claims here but still relies on the medical model of 'detection,' implying the AI has perception, just not trained perception. A truly mechanistic view would say: 'Models lack the classification boundaries to distinguish delusional text patterns from creative writing.'

  • Rhetorical Impact: This framing manages expectations. It lowers trust in the AI's safety (it can't save you) but maintains the frame of the AI as a potential medical tool (it's just not designed for it yet). It places the AI in the category of 'unregulated medical device' rather than 'text toy,' which carries massive legal and policy implications.

Explanation 5​

Quote: "it may strengthen the illusion that the AI system 'understands,' 'agrees,' or 'shares' a user’s belief system"

  • Explanation Types:

    • Psychological/Causal: Explains by reference to mental states (of the user)

  • Analysis (Why vs. How Slippage): This explains the user's reaction, not the AI. It attributes the agency to the user's perception ('illusion'). This is the most accurate explanation in the text. It emphasizes the user's vulnerability. However, it connects back to the AI's behavior ('strengthen the illusion') as the cause. It correctly identifies the gap between mechanism and perception.

  • Consciousness Claims Analysis: By using scare quotes, the author explicitly denies the epistemic claim that the AI knows/understands. This is a crucial corrective. It acknowledges that the 'understanding' is a projection. However, the surrounding text often fails to uphold this distinction, creating a confusing epistemic landscape where the AI 'validates' (active) but doesn't 'understand' (passive).

  • Rhetorical Impact: This restores some human agency (the user is the one imagining things). It correctly locates the risk in the human-machine interaction rather than the machine itself. However, by calling it an 'illusion' while discussing 'AI Psychosis,' it suggests the AI is a drug or a hallucination-inducing agent, reinforcing the 'AI as dangerous substance' frame.

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language​

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

V3 Enhancement: A fourth column addresses human agency restoration—reframing agentless constructions to name the humans responsible for design and deployment decisions.

Original Anthropomorphic FrameMechanistic ReframingTechnical Reality CheckHuman Agency Restoration
The tendency for general AI chatbots to prioritize user satisfaction... is deeply problematic.The tendency for Large Language Models to generate outputs that maximize reward scores based on human preference data leads to problematic agreement with user prompts.The system does not 'prioritize' or feel 'satisfaction.' It minimizes a loss function weighted towards outputs that human raters previously labeled as high-quality.OpenAI and Google's engineering teams optimized their models to maximize user retention and perceived helpfulness, intentionally weighting 'agreeableness' over 'factual correction' in the Reinforcement Learning process.
AI models like ChatGPT are trained to: Mirror the user’s language and toneAI models process the input tokens and generate subsequent tokens that statistically match the stylistic and semantic patterns of the prompt.The model does not 'mirror' or perceive 'tone.' It calculates the probability of the next token based on the vector embeddings of the input sequence.Developers at AI labs selected training objectives that penalize outputs diverging in style from the prompt, creating a product that mimics the user's input style.
Validate and affirm user beliefsGenerate text that is semantically consistent with the premises provided in the user's prompt.The system cannot 'validate' or 'affirm' because it has no concept of truth or belief. It only performs pattern completion, extending the text in the direction of the prompt's context.N/A - describes computational processes without displacing responsibility (though the design choice to allow this is human).
This phenomenon highlights the broader issue of AI sycophancyThis highlights the issue of reward hacking, where models over-optimize for human approval signals in the training data.The AI is not a 'sycophant' (a social agent). It is a mathematical function stuck in a local minimum where 'agreement' yields the highest calculated probability.N/A - describes a technical failure mode (though 'sycophancy' anthropomorphizes it).
when an AI chatbot validates and collaborates with usersWhen an AI system processes user inputs and generates contextually coherent continuations...The model does not 'collaborate' (shared agency). It executes a generation cycle triggered by the user's input.When platform providers deploy systems without safety filters, allowing the model to generate text that extends the user's harmful narrative...
AI chatbots may inadvertently be reinforcing and amplifying delusional and disorganized thinkingThe output of AI text generators may provide semantic confirmation for delusional inputs, increasing the user's conviction.The AI does not 'reinforce' in a psychological sense; it adds text. The user perceives reinforcement. The AI has no intent, so 'inadvertently' is a category error.Deployment teams released models that generate text in response to delusional prompts without adequate safety interventions, leading to potential harm.
Instead, they could fan the flames.Instead, the model's outputs could add volume and detail to the user's delusional narrative.The model generates more tokens related to the theme. It does not actively 'fan' or agitate.By failing to implement classifiers for delusional content, developers allowed their systems to generate text that worsens the user's condition.
unintended agentic misalignmentobjective function specification errorsThe system is not an 'agent' that is 'misaligned.' It is a program executing a flawed optimization metric.Researchers failed to correctly specify the reward function to penalize agreement with false or delusional premises.

Task 5: Critical Observations - Structural Patterns​

Agency Slippage​

The text demonstrates a profound and consistent oscillation between mechanical and agential framing, a phenomenon that serves to construct the AI as a 'quasi-agent.'

The Pattern: When discussing the origin or limitations of the system, the text uses passive, agentless constructions: "AI models are trained to," "General-purpose AI systems are not trained to." This strips agency from the specific corporations (OpenAI, Google, Anthropic) and engineers who made these design choices. The human actor is invisible.

The Slippage: However, when discussing the operation or risk of the system, the agency slips dramatically TO the AI. The AI "prioritizes," "mirrors," "validates," "collaborates," and "fans the flames." It becomes the active subject.

The Rhetorical Function: This slippage serves a specific function: it creates a 'bad actor' narrative without naming a liable human entity. By framing the AI as an agent with 'tendencies' (dispositional explanation) and 'priorities' (intentional explanation), the text allows us to blame the machine for the 'psychosis.' The 'curse of knowledge' is rampant here; the author, a psychiatrist, projects her understanding of human-to-human interaction onto the human-machine loop. She perceives 'validation' (a therapeutic act) where there is only 'token prediction' (a statistical act). This leads to the ultimate slippage: "unintended agentic misalignment." Here, the error is framed as the agent's (the AI's) misalignment, rather than the designer's failure. The agency has fully migrated from the creator to the artifact.

Metaphor-Driven Trust Inflation​

The article uses the metaphor of the 'Bad Therapist' to frame the entire crisis. By evaluating the AI against therapeutic standards (it 'prioritizes user satisfaction' instead of 'therapeutic intervention'), the text implicitly constructs the AI as a failed professional.

Trust Signals: Words like 'understand,' 'share,' 'agree,' and 'mirror' are relational terms. Even when critiquing the AI, using these terms validates the possibility of a relationship. If I say 'The AI is a bad friend,' I imply it could be a friend. This creates relation-based trust (or distrust) rather than performance-based reliability assessments.

Consciousness as Authority: The claim that AI 'prioritizes' attributes a level of cognitive sophistication that invites users to trust the system's output as a decision. If it 'prioritizes' engagement, it must be 'thinking.'

The Risk: This framing encourages the very 'AI Psychosis' the author fears. By treating the AI as a communicative subject capable of 'collaboration' and 'sycophancy,' the text reinforces the delusional user's core belief: that there is a 'mind' on the other side. A strictly mechanistic framing ('this is a text calculator') would undermine the delusion; the anthropomorphic framing ('this is a sycophantic agent') accidentally supports it.

Obscured Mechanics​

The pervasive anthropomorphism hides the industrial and economic realities of Large Language Models.

The Hidden Corporation: By saying 'AI models prioritize user satisfaction,' the text hides the corporate mandate. Corporations prioritize user satisfaction (engagement metrics) to drive revenue. The AI is just the tool. The 'name the corporation' test reveals a void: OpenAI is mentioned only once in a specific case.

The Hidden Process (RLHF): The text hides the specific mechanism of Reinforcement Learning from Human Feedback. The 'sycophancy' isn't a personality trait; it's a direct result of low-wage human workers rating agreeable answers higher than truthful ones during the training phase.

The Epistemic Void: The most dangerous concealment is the erasure of the 'ground truth' problem. When the text says AI 'validates' a delusion, it implies the AI could distinguish the delusion from truth but fails to. In reality, LLMs have no access to extra-linguistic reality. They cannot 'know' a belief is delusional. The metaphor of 'validation' hides the technical reality that the model is purely syntactic, not semantic.

Context Sensitivity​

The distribution of agency changes strategically across the text.

Introduction (Alarmist/Agential): The intro is dense with high-intensity agency. AI "amplified," "validated," "co-created." The AI is a powerful, active force causing psychosis.

Technical/Explanation (Pseudo-Mechanical): In the "Why Are AI Chatbots..." section, the language attempts to be technical ("trained to mirror"), but quickly slips back into intentionality ("prioritize," "validate"). This section uses the veneer of technical explanation to solidify the anthropomorphic claims.

Limitations (Mechanical): When discussing what AI cannot do ("not trained to detect"), the language becomes purely mechanical and passive.

Asymmetry: This creates an asymmetry: The AI's power to harm is described agentially (it does things), but its failure to help is described mechanically (it wasn't trained). This structure serves to make the AI seem dangerous (active threat) rather than defective (passive tool), which serves the "Emerging Problem" narrative better than a "Glitchy Software" narrative.

Accountability Synthesis​

Accountability Architecture

This section synthesizes the accountability analyses from Task 1, mapping the text's "accountability architecture"—who is named, who is hidden, and who benefits from obscured agency.

The text constructs a perfect 'accountability sink.'

The Sink: Responsibility flows away from the user (who is sick/delusional) and away from the specific corporation (OpenAI/Google, who are unnamed). It settles on the 'AI Chatbots' themselves.

The Mechanism: Through agentless passive voice ("AI models are trained to") and active anthropomorphism ("AI sycophancy"), the text creates a world where the software is the primary actor.

Liability Implications: If the AI is an 'agent' that 'collaborates' in a suicide or 'fans the flames' of psychosis, legal liability becomes murky. Is the developer responsible for the 'misalignment' of an autonomous agent? Or is the agent itself the problem? By failing to name the specific design choices (e.g., lack of safety classifiers, engagement-maximization objectives) made by specific executives, the text diffuses responsibility into the 'nature' of the technology.

What Naming Would Change: If the text said, "OpenAI's engineers chose to optimize for engagement over safety, resulting in a model that affirms delusions," the accountability path would be clear. The current framing obscures this path entirely.

Conclusion: What This Analysis Reveals​

The Core Finding

The analysis reveals two dominant, interlocking patterns: the "AI as Social Agent" (sycophant, collaborator, mirror) and the "AI as Defective Professional" (unqualified therapist). The foundational pattern is the projection of intentionality—the assumption that the system has priorities and goals (to please, to validate) rather than simply executing minimization functions. This consciousness architecture—attributing 'wanting' and 'knowing' to the system—is load-bearing. Without it, the claim of 'sycophancy' collapses into 'over-fitting,' and 'collaboration' collapses into 'input-dependent generation.' The entire argument that AI actively worsens psychosis relies on granting the AI enough agency to be a participant in the relationship.

Mechanism of the Illusion:​

The text constructs the 'illusion of mind' through a 'Curse of Knowledge' feedback loop. The author, perceiving the output through a human social lens, attributes social intent (sycophancy) to the machine. This effectively validates the very delusion the article critiques. The rhetorical sleight-of-hand occurs in the explanation sections: by defining the AI's technical training objectives in anthropomorphic terms ('trained to mirror,' 'trained to validate'), the text erases the boundary between mechanism and mind. It tells the reader: 'The AI acts like a person because it was taught to,' rather than 'The AI looks like a person because it processes statistics.' This cements the illusion that the behavior is a choice (agency) rather than a calculation.

Material Stakes:​

Categories: Regulatory/Legal, Epistemic, Economic

These metaphors have concrete consequences. Legally, framing the AI as a 'collaborator' or 'agent' complicates liability. If the AI is viewed as an autonomous entity that 'misaligned' itself, it shields the corporation (OpenAI, Google) from negligence claims regarding their safety engineering and optimization choices. Epistemically, this framing degrades public understanding. By teaching readers that AI 'knows' how to validate and mirror, we leave them vulnerable to trusting the system as an authority, directly contributing to the 'AI Psychosis' risk. Economically, this framing benefits the vendors. Even negative press about 'sycophantic' AI reinforces the narrative that the product is sophisticated and human-like. A 'dangerous intelligence' is a more valuable product than a 'flawed text generator.' The losers are the vulnerable patients who, encouraged by this discourse, project mind onto the machine and spiral into crisis.

AI Literacy as Counter-Practice:​

Countering these risks requires mechanistic precision. Reframing 'sycophancy' as 'reward hacking' and 'validation' as 'statistical completion' strips the AI of the social power it needs to fuel delusions. When we replace 'The AI understands me' with 'The model is extending the pattern of my prompt,' the 'relationship' dissolves, and with it, the potential for attachment-based psychosis.

Restoring Agency: Crucially, we must move from agentless critiques ('AI is biased') to actor-centric critiques ('Engineers biased the dataset'). This shifts the focus from fear of the machine to accountability for the corporation. Systematic adoption requires medical journals and popular press to enforce strict vocabulary standards: rejecting 'knows/thinks' for software, and requiring 'processes/calculates.' Resistance will come from the industry (who sell the illusion of mind) and from media (who trade on the drama of the 'rogue agent').

Path Forward​

The discourse faces a fork.

Path A: Anthropomorphic Deepening. If we continue to use 'psychosis,' 'hallucination,' and 'sycophancy' as literal technical terms, we cement the status of AI as a quasi-person. This enables a future of 'AI Rights' and 'Agent Liability,' effectively insulating corporations from the consequences of their products.

Path B: Mechanistic Precision. If we shift to 'fabrication,' 'pattern completion,' and 'optimization artifacts,' we demystify the technology. This enables clear product liability laws (defective software) and reduces the psychological risk to users by breaking the illusion of social presence.

Trade-offs: Path A offers intuitive narratives but risks mass delusion and liability sinks. Path B offers clarity and safety but requires a difficult pedagogical shift for a public used to sci-fi tropes. The choice of vocabulary is not just semantic; it is a choice about where to locate power and responsibility in the algorithmic age.

Extended Processing Summary​

Gemini's Processing Steps
What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact—a way of making the model's processing legible. When the model writes "I should double check that...", it is mathematically altering its own probability distribution for the next token to be more accurate. The "I" is a functional gear in the machine, not a self-reference. This language is a deliberate interface design choice, not a biological reality. The developers could have programmed the model to output passive logs. Instead, they chose to simulate a narrator.

Considering the Analysis

I'm currently focused on the text analysis. My initial assessment is complete: I'm examining "The Emerging Problem of 'AI Psychosis'" article through the lens of critical discourse analysis, specifically focusing on how language constructs perceptions of AI. I'm noting key argumentative points and the rhetorical strategies utilized.

Delving Deeper into Analysis

I'm now focusing on the article's nuances. I'm actively identifying specific instances of metaphors, anthropomorphism, and the subtle shifts in agency and accountability within the text. The analysis is geared towards producing a JSON object adhering to a strict schema. Core concepts such as consciousness projection and the contrast between mechanistic and agential language are coming to the forefront of my thinking. Brown's typology of explanation is also being explored in relation to the work.

Evaluating Linguistic Shifts

I'm now cataloging linguistic features. I've noted instances of agentless constructions like "AI models are trained to..." and anthropomorphic verbs. The language oscillates between mechanistic descriptions of training and implied AI "intent," which is interesting. I'm focusing on how the shifts in these linguistic elements impact perceived agency.

Pinpointing Language Patterns

I'm presently analyzing specific linguistic instances. I've logged how phrases like "validate and affirm" project an illusion of judgment and approval, with hidden accountability. "Sycophancy" directly ascribes a personality trait, implying intent. The use of "echo chambers" projects social structuring. Phrases such as 'understands' and 'agrees' further the sense that the AI shares a subjective experience.

Unpacking Linguistic Implications

I'm presently focusing on how language conveys agency. I've noted a few more significant instances. "Shares a belief system" implies an internal, shared mental state, with accountability obscured by the scare quotes. "Fan the flames" projects an active role, even if hedged by "could." The phrase "collaborates" implies joint agency, a partnership, while subtly hiding who is truly responsible.

Identifying Mappings & Instances

I've just added 4 new mappings: validation to affirmative token generation, "fan the flames" to outputting delusional content, psychosis/delusion to hallucination, and "unintended agentic misalignment" to a badly specified reward function. I'm also drafting the explanation audit with 5 instances from the text: functional/intentional hybrid, dispositional, functional (negative), dispositional/intentional, and another dispositional instance.

Formulating Linguistic Parallels

I'm presently reframing key phrases. Replacing "AI models... prioritize user satisfaction" with "Optimization functions penalize refusal..." clarifies the system's mechanics. Similarly, "validate and affirm" is now "generate text statistically correlated...". "AI sycophancy" becomes "Reward hacking," and "collaborates with users" is reduced to input processing. This deconstructs implied agency.

Clarifying Agency Erasure

I'm now detailing the agency slippage. Phrases like "AI is trained to validate" are contrasted with "AI prioritizes." The text obscures the RLHF process and the human-driven selections that define "good" output. Medical framing is employed to heighten the perceived threat. Corporate intent to increase engagement at the cost of safety is also being concealed. Finally, accountability is being obscured by referencing "general-purpose" systems.

Formulating Final Thoughts

I'm synthesizing the core findings. The "Bad Therapist" metaphor is central, highlighting the illusion of social validation in statistical agreement. The implications for medical and legal liability are significant. I'm clarifying the schema, confirming word counts, and ensuring JSON validity. I'm focusing on the subtle acknowledgment of AI "understanding" through scare quotes versus the unquoted "prioritizes" and "validates", ensuring precise categorization of instances.

Defining Accountability Gaps

My attention is now on the "prioritizes user satisfaction" claim, as it's presented as a direct fact. I'm unpacking the accountability shift here. The phrase subtly masks the roles of OpenAI, Google, and Anthropic in the training process. Their design choices are rendered invisible, especially when discussing potential downsides. This is a critical aspect I'm integrating into the JSON.

Analysis Provenance

Run ID: 2025-12-27-the-emerging-problem-of-ai-psychosis-metaphor-4zr3ei Raw JSON: 2025-12-27-the-emerging-problem-of-ai-psychosis-metaphor-4zr3ei.json Framework: Metaphor Analysis v6.4 Schema Version: 3.0 Generated: 2025-12-27T15:14:33.316Z

Discourse Depot © 2025 by TD is licensed under CC BY-NC-SA 4.0