Skip to main content

🆕 The application of large language models (LLMs) in psychological support for university students: A scoping review

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping), the philosophy of social science (Robert Brown's typology of explanation), and accountability analysis.

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.

Task 1: Metaphor and Anthropomorphism Audit​

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

V3 Enhancement: Each metaphor now includes an accountability analysis.

1. Computational Processing as Cognitive Comprehension​

Quote: "These models possess an unprecedented capacity for natural language understanding..."

  • Frame: Model as conscious comprehender
  • Projection: This metaphor projects the human cognitive faculty of conscious comprehension onto statistical pattern-matching algorithms. By using the word "understanding," the text implies that the AI system forms internal mental models, grasps semantic meaning, and possesses a subjective awareness of the text it processes. In reality, Large Language Models merely calculate the statistical probabilities of token sequences based on their training data. They process text without knowing what the words mean. This projection fundamentally blurs the line between a system that accurately predicts string continuations and an agent that actively comprehends a human user's emotional or psychological state.
  • Acknowledgment: Direct (Unacknowledged) (The text states "capacity for natural language understanding" as a literal, unhedged capability. I considered "Hedged/Qualified" because it appears alongside "generation" (a technical term), but there is no qualification indicating that "understanding" is used merely in a functional or metaphorical sense.)
  • Implications: Attributing conscious comprehension to an AI system in a clinical or psychological context creates dangerous over-trust. If university administrators and students believe the system "understands" them, they will assume it can evaluate contextual nuances, recognize genuine distress, and provide medically justified responses. This consciousness projection inflates the system's perceived clinical sophistication, obscuring its reliance on probabilistic correlation and raising the risk of inappropriate deployment in complex mental health crises.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The grammatical subject is "These models," effectively erasing the human actors. The developers at companies like OpenAI who designed the algorithms, selected the training data, and aligned the models are hidden. Furthermore, the universities purchasing and deploying these tools are obscured. By framing the models as possessing innate capacities, the text shields these corporate and institutional actors from liability if the system fails to parse a student's suicidal ideation correctly. I considered "Partial" since the broader field implies developers, but ruled it out because this specific sentence entirely displaces agency onto the artifact itself.
Show more...

2. Algorithmic Routing as Professional Triage​

Quote: "...where AI handles routine support and escalates complex issues to human counselors."

  • Frame: AI as autonomous clinical practitioner
  • Projection: This phrasing projects the professional judgment, intentionality, and situational awareness of a human clinician onto a software routing mechanism. The verbs "handles" and "escalates" imply conscious decision-making, where an agent actively evaluates a patient's severity, recognizes the limits of its own expertise, and intentionally transfers care to a superior. In reality, the system merely triggers a pre-programmed classification threshold based on keyword matching or sentiment scoring. It does not "escalate" out of caution or care; it executes a conditional computational loop without any subjective grasp of the "complex issues" it is processing.
  • Acknowledgment: Direct (Unacknowledged) (The actions "handles" and "escalates" are presented as literal operations of the AI without quotes or caveats. I considered "Hedged/Qualified" because the surrounding context discusses "human-in-the-loop" frameworks, but the specific verbs attribute independent, unqualified agency directly to the AI.)
  • Implications: This framing lulls institutional stakeholders into a false sense of security. By describing the system as an autonomous agent capable of "handling" and "escalating," universities may over-rely on the software for crisis triage. Believing the AI "knows" when an issue is complex encourages institutions to reduce human oversight, potentially abandoning students in acute distress to rigid, brittle statistical thresholds that lack genuine clinical judgment.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The AI is presented as the sole actor managing triage. The software engineers who hardcoded the keyword triggers, the data annotators who labeled the severity thresholds, and the university administrators who approved these risk parameters are completely erased. If a student's crisis is missed, the framing suggests the "AI failed to escalate," rather than "the university's risk-detection parameters were inadequately designed." I considered "Partial" because "human counselors" are mentioned, but ruled it out because the active decision-making agency is placed entirely on the AI.

3. Context Window Limits as Biological Memory Loss​

Quote: "Technical and Functional Issues: Bugs, slow response times, rigid conversation flows, and poor memory (forgetting previous conversations)..."

  • Frame: System limitation as human cognitive failure
  • Projection: The text maps human biological memory—specifically the cognitive act of "forgetting"—onto a software system's token context window and database architecture. Humans forget because of cognitive decay, distraction, or psychological factors; an LLM "forgets" because its session parameters mathematically truncate earlier tokens to fit within hardware processing limits, or because backend databases fail to retrieve prior embeddings. Attributing a "poor memory" to a machine anthropomorphizes a purely technical constraint, suggesting the AI is a conscious entity experiencing a lapse in cognitive continuity.
  • Acknowledgment: Direct (Unacknowledged) (The text uses "poor memory" and explicitly defines it as "forgetting previous conversations" without any meta-commentary indicating this is a functional metaphor for context limits. I considered "Explicitly Acknowledged" because it lists this under "Technical Issues," but the parenthetical literalizes the anthropomorphism.)
  • Implications: Framing technical constraints as biological memory flaws masks the material realities of the technology (e.g., the high financial and computational costs of maintaining long context windows). For users, assuming the AI "forgets" encourages them to treat it like a flawed human rather than recognizing that they are interacting with an amnesiac statistical generator that requires specific prompting strategies to maintain state.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The text attributes the "forgetting" to the system's own intrinsic "poor memory." This obscures the developers who purposefully restricted the context window to save computing costs, and the designers who failed to implement persistent vector databases for conversation retrieval. By naturalizing this as a memory flaw, it avoids naming the corporations making economic trade-offs regarding computational resources. I considered "Ambiguous" due to the bulleted list format, but the semantic displacement of agency onto the software's "memory" is clear.

4. Statistical Deviation as Subjective Misapprehension​

Quote: "A salient concern was the AI's potential to misunderstand user statements, provide inappropriate advice, fail to detect or adequately respond to crisis situations..."

  • Frame: AI as an interpreting conversationalist
  • Projection: This passage projects the human capacity for subjective misinterpretation onto an algorithmic classification error. "Misunderstand" implies an entity that forms an incorrect mental model of an utterance but remains a conscious receiver. An AI cannot "misunderstand" because it never "understands" in the first place; it processes vector embeddings. Similarly, "fail to detect" implies an agent that was looking but missed something, projecting an attentional state onto a passive statistical classifier that merely failed to correlate a user's input with a predefined crisis threshold.
  • Acknowledgment: Direct (Unacknowledged) (The capabilities to "misunderstand" and "fail to detect" are attributed as direct potentials of the AI. I considered "Hedged/Qualified" because it is framed as a "salient concern" and a risk, but the cognitive verbs themselves are applied literally to the AI without qualification.)
  • Implications: This projection is fundamentally epistemic: it assumes the AI generally possesses "understanding" and only occasionally "misunderstands." This logic prevents users and clinicians from realizing that the system is always guessing. It creates a paradigm where AI errors are viewed as human-like mistakes rather than profound, systemic inabilities to process meaning. This shifts the focus from the inherent unreliability of statistical generation to trying to "correct" a supposedly capable agent.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The text frames the failures as "the AI's potential" to do harm. The human developers who failed to include sufficient crisis-detection parameters in the training data, and the university administrators who deployed a fundamentally unreliable system to vulnerable students, are erased. The risk belongs to the AI, making it the liability sink. I considered "Named" since users are mentioned, but the entity responsible for the failure is solely identified as "the AI."

5. Statistical Tone as Emotional Resonance​

Quote: "While all three modalities reduced stress, the virtual human and chatbot were less empathetic but achieved better homework adherence..."

  • Frame: AI as an emotional being
  • Projection: This maps the human capacity for affective empathy—the ability to share and comprehend another's emotional state—onto the statistical generation of sympathetic-sounding text. Describing a chatbot as "less empathetic" suggests it possesses an internal emotional reservoir that is simply smaller than a human's. In reality, a chatbot has zero empathy; it generates outputs that human users perceive as having a particular tone based on correlations in its training corpus. The metaphor conflates the simulation of an emotion with the capacity to experience it.
  • Acknowledgment: Direct (Unacknowledged) (The text directly states the chatbot was "less empathetic" as an empirical finding. I considered "Hedged/Qualified" because another section mentions "simulated empathetic interaction," but in this specific results synthesis, the empathy is attributed as a literal property.)
  • Implications: Treating simulated empathy as actual empathy invites vulnerable university students into parasocial relationships with statistical systems. When users believe a machine "feels" for them, they apply relation-based trust (expecting sincerity and care) rather than performance-based trust (expecting reliable processing). This can lead to profound psychological harm if the system subsequently outputs a "hallucinated" toxic response, betraying a trust it never actually possessed.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The chatbot is treated as an autonomous entity whose internal emotional limits result in being "less empathetic." The developers who tuned the reinforcement learning from human feedback (RLHF) to optimize for task-completion over conversational warmth, or the researchers who engineered the prompts, are completely invisible. I considered "Partial" since the broader sentence mentions a "human therapist" comparison, but the responsibility for the digital artifact's behavior is displaced solely onto the artifact.

6. Correlative Mechanics as Logical Cognition​

Quote: "No study investigated how explaining the AI's reasoning (Explainable AI - XAI) might affect user trust..."

  • Frame: Algorithm as logical thinker
  • Projection: The text projects human deductive logic, rational justification, and deliberate thought processes onto complex, opaque mathematical operations (matrix multiplications in a neural network). The term "reasoning" implies that the AI holds justified beliefs and arrives at conclusions through a conscious, step-by-step evaluation of evidence. In truth, LLMs do not reason; they calculate the highest probability distribution for the next token based on billions of parameters. XAI attempts to map the mathematical weights that led to an output, which is a mechanistic tracing of variables, not the recovery of a cognitive rationale.
  • Acknowledgment: Direct (Unacknowledged) (The phrase "the AI's reasoning" is presented as a literal, existing phenomenon that needs to be explained. I considered "Explicitly Acknowledged" due to the inclusion of the acronym "(Explainable AI - XAI)", but the parenthetical acts as an academic label, not as a critique or hedge of the word "reasoning.")
  • Implications: Calling mathematical optimization "reasoning" drastically inflates the epistemic authority of the system. It suggests that if an AI recommends a specific cognitive restructuring exercise to a depressed student, it did so because it logically deduced it was the best course of action. This obscures the reality that the recommendation is merely a statistical artifact, making it harder for clinicians to challenge algorithmic outputs and increasing the likelihood of automation bias.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: By locating "reasoning" inside the AI, the text obscures the human actors who designed the architecture and selected the training data that govern these outputs. The demand for "explaining the AI's reasoning" frames the machine as a complex mind to be deciphered, rather than demanding that tech companies (OpenAI, Google) explain their proprietary algorithms, data curation practices, and corporate design choices. I considered "Partial" because the sentence refers to future research practices, but the active processing agent is entirely the AI.

7. Societal Data Disparities as Algorithmic Prejudice​

Quote: "Few studies discussed, let alone measured, potential biases... in the LLM's responses, which risks perpetuating inequalities in care."

  • Frame: Model as prejudiced actor
  • Projection: This maps the human social flaw of prejudice (bias) onto the output distribution of a machine learning model. While algorithmic bias is a standard technical term, phrasing it as "biases... in the LLM's responses" subtly frames the system as an independent actor that harbors and enacts prejudice. It obscures the fact that the system merely reflects, processes, and amplifies the statistical distribution of historical human biases, systemic racism, and exclusionary language present in the datasets compiled by humans.
  • Acknowledgment: Hedged/Qualified (I chose "Hedged/Qualified" because the text contextualizes this in another sentence as "LLMs are trained on datasets that can perpetuate societal biases," acknowledging the mechanical origin of the issue. I considered "Direct" for the quoted sentence in isolation, but the surrounding paragraph clearly explains the dataset dependency.)
  • Implications: While highlighting bias is critical, attributing it primarily to "the LLM's responses" can inadvertently position the technology as a rogue actor acting unfairly, rather than treating the system as a mirror of institutionalized human inequalities. This can lead to purely technical "de-biasing" solutions (tweaking weights) rather than confronting the social, historical, and economic contexts of the data collection and the systemic inequalities of the healthcare system.

Accountability Analysis:

  • Actor Visibility: Partial (some attribution)
  • Analysis: The text notes that LLMs are trained on datasets reflecting societal biases, pointing vaguely to the human origins of the problem. However, it still falls short of naming the specific tech corporations that scrape, exploit, and fail to curate these datasets, or the underpaid gig workers forced to annotate them. The responsibility is diffused into abstract "datasets" and "societal biases." I considered "Hidden" because specific actors aren't named, but "Partial" is more accurate as it acknowledges systemic human origins.

8. Algorithmic execution as biological behavior​

Quote: "Investigate how transparency about the AI's functioning (XAI) impacts therapeutic alliance..."

  • Frame: System operations as agential functioning
  • Projection: The text projects the idea of an independent, biological, or autonomous organism "functioning" onto the execution of computer code. "Functioning" implies an ongoing, self-directed state of being, much like a human mind or body functions. Mechanistically, an AI does not "function" in a holistic or biological sense; it passively executes scripts, retrieves data, processes matrix multiplications when prompted, and returns an output. Projecting "functioning" onto the model obscures the discrete, contingent, and human-prompted nature of its operations.
  • Acknowledgment: Direct (Unacknowledged) (The "AI's functioning" is stated as a straightforward operational fact. I considered "Explicitly Acknowledged" due to the (XAI) tag, but the phrase lacks any explicit signifier that it is metaphorical.)
  • Implications: This framing normalizes the treatment of software as a quasi-living participant in a "therapeutic alliance." If users and researchers believe they are interacting with a continually "functioning" entity, they are more likely to project consciousness, internal consistency, and reliability onto it. This drastically complicates the ethics of digital mental health, as it encourages patients to bond with an illusion.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The AI is the sole owner of its "functioning." The tech companies that own the servers, write the APIs, set the system prompts, and throttle the processing power are completely invisible. By focusing on "the AI's functioning," the text sidesteps the need for transparency about the corporate supply chain, data harvesting, and specific engineering decisions that actually dictate how the software behaves. I considered "Named" since researchers are implicitly tasked with investigating, but the agency of the operation belongs to the AI.

Task 2: Source-Target Mapping​

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: A conscious human mind capable of grasping semantic meaning, context, and intent. → Large Language Models utilizing transformer architectures for next-token prediction.​

Quote: "These models possess an unprecedented capacity for natural language understanding..."

  • Source Domain: A conscious human mind capable of grasping semantic meaning, context, and intent.
  • Target Domain: Large Language Models utilizing transformer architectures for next-token prediction.
  • Mapping: The metaphor maps the subjective, conscious experience of "understanding" onto the mathematical processing of language data. It invites the assumption that when the model ingests text about a student's depression, it internally comprehends the emotional weight, real-world context, and semantic meaning of the words, just as a human clinician would. This consciousness mapping suggests that the output is generated because the system "knows" what is being discussed and has formed a justified response to that reality.
  • What Is Concealed: This mapping conceals the purely mechanistic, statistical nature of token prediction. It hides the fact that the system relies entirely on vector embeddings and attention weights, possessing zero subjective awareness or semantic grounding in the real world. Furthermore, it obscures the proprietary opacity of models like GPT-4; researchers cannot actually observe "understanding," they can only observe statistically plausible text outputs generated by a black-box system controlled by a private corporation.
Show more...

Mapping 2: A professional triage nurse or junior clinician making active judgments. → An automated algorithmic routing system based on keyword matching or sentiment analysis.​

Quote: "...where AI handles routine support and escalates complex issues to human counselors."

  • Source Domain: A professional triage nurse or junior clinician making active judgments.
  • Target Domain: An automated algorithmic routing system based on keyword matching or sentiment analysis.
  • Mapping: The relational structure of a clinical hierarchy—where a junior agent evaluates a case, determines their own limitations, and actively hands it over to a senior expert—is mapped onto a software program's conditional logic. It projects intentionality, caution, and professional judgment onto the machine, suggesting that the AI actively protects the patient by choosing to "escalate" when it realizes a situation is beyond its "competence."
  • What Is Concealed: This conceals the rigid, threshold-based reality of algorithmic routing. The system does not "know" an issue is complex; it merely triggers a pre-programmed "IF/THEN" escalation protocol when specific tokens (e.g., "suicide") are detected. It hides the vulnerability of this mechanism to slight phrasing variations, the absence of actual clinical judgment, and the liability of the human developers who set those often-flawed parameters.

Mapping 3: A human being experiencing cognitive decay or forgetfulness. → A software application's context window limits or database retrieval failures.​

Quote: "Technical and Functional Issues: Bugs, slow response times, rigid conversation flows, and poor memory (forgetting previous conversations)..."

  • Source Domain: A human being experiencing cognitive decay or forgetfulness.
  • Target Domain: A software application's context window limits or database retrieval failures.
  • Mapping: The structure of human memory loss—a biological, often involuntary lapse where an entity fails to recall an event it previously experienced—is mapped onto a system's inability to reference prior user inputs. It invites the assumption that the chatbot is a continuous conversational partner that "tried" to remember but failed due to an internal, quasi-cognitive flaw.
  • What Is Concealed: This mapping completely obscures the material, computational realities of LLM infrastructure. It hides the fact that "forgetting" is actually a hardcoded limitation of the model's token context window, a design choice made by engineers to manage computational costs. It conceals the absence of a persistent identity in the machine; every prompt is processed statelessly from scratch, a mechanistic reality completely unlike human memory.

Mapping 4: An attentive, conscious listener who misinterprets meaning or misses a cue. → A statistical classifier failing to map a user's input to the desired output vector.​

Quote: "A salient concern was the AI's potential to misunderstand user statements, provide inappropriate advice, fail to detect or adequately respond to crisis situations..."

  • Source Domain: An attentive, conscious listener who misinterprets meaning or misses a cue.
  • Target Domain: A statistical classifier failing to map a user's input to the desired output vector.
  • Mapping: This projects the psychological state of misapprehension onto mathematical misclassification. By suggesting the system can "misunderstand" or "fail to detect," it maps the image of a conscious entity that is trying to comprehend a situation onto a machine. It invites the assumption that the system generally possesses awareness and merely makes occasional human-like interpretive errors.
  • What Is Concealed: This deeply conceals the fact that the system never "understands" anything. It obscures the reality that what we call "misunderstanding" is actually the model generating statistically probable but contextually incorrect tokens because the user's phrasing didn't align strongly enough with the training data distribution. It hides the lack of true causal modeling, grounding the error in a false paradigm of cognitive failure rather than a lack of actual cognition.

Mapping 5: An unfeeling or emotionally stunted human being. → An AI text generator optimized for therapeutic frameworks but lacking specific affective prompt tuning.​

Quote: "While all three modalities reduced stress, the virtual human and chatbot were less empathetic but achieved better homework adherence..."

  • Source Domain: An unfeeling or emotionally stunted human being.
  • Target Domain: An AI text generator optimized for therapeutic frameworks but lacking specific affective prompt tuning.
  • Mapping: The human emotional spectrum—ranging from highly empathetic to cold and clinical—is projected onto a software artifact. It maps the capacity for emotional resonance onto the mere generation of text. This assumes that "empathy" is a measurable substance or trait residing within the machine, rather than a subjective experience generated entirely in the mind of the human user based on the simulation of caring language.
  • What Is Concealed: This mapping conceals the total absence of internal subjective experience in the machine. It hides the mechanistic reality that a chatbot isn't "less empathetic"; it merely lacks the prompt engineering or RLHF tuning necessary to output tokens that mimic human warmth. By literalizing simulated empathy, it also obscures the ethical danger of deceiving users into believing a machine cares for them.

Mapping 6: A rational human thinker formulating justified arguments and logical deductions. → The billions of parameter weights and activation patterns inside a neural network.​

Quote: "No study investigated how explaining the AI's reasoning (Explainable AI - XAI) might affect user trust..."

  • Source Domain: A rational human thinker formulating justified arguments and logical deductions.
  • Target Domain: The billions of parameter weights and activation patterns inside a neural network.
  • Mapping: This maps the human process of logical deduction—arriving at a conclusion through conscious evaluation of facts and rules—onto the opaque, multidimensional vector mathematics of an LLM. It invites the assumption that the AI "thinks" its way to an answer, possessing an internal, logically sound rationale for its outputs that can simply be "explained" or translated to the user.
  • What Is Concealed: This obscures the fundamental nature of neural networks, which operate via statistical correlation, not deductive logic or causal reasoning. It conceals the fact that LLMs are black boxes owned by private corporations, making true epistemic transparency virtually impossible. XAI doesn't reveal "reasoning"; it highlights statistical weight distributions. The metaphor exploits rhetorical confidence to hide profound technical opacity.

Mapping 7: A biased, prejudiced human individual making unfair judgments. → A mathematical model reproducing statistical disparities present in its training corpus.​

Quote: "Few studies discussed, let alone measured, potential biases... in the LLM's responses, which risks perpetuating inequalities in care."

  • Source Domain: A biased, prejudiced human individual making unfair judgments.
  • Target Domain: A mathematical model reproducing statistical disparities present in its training corpus.
  • Mapping: The structure of human prejudice—where an individual harbors conscious or unconscious animus toward a group—is mapped onto an algorithm's output distribution. This invites the assumption that the machine itself possesses a worldview and actively discriminates when it generates responses.
  • What Is Concealed: This mapping conceals the massive, exploitative, and unregulated human labor and data scraping that created the training corpus. It hides the tech companies (e.g., OpenAI) that indiscriminately Hoovered up the racist, sexist, and biased text of the internet, as well as the low-paid annotators who reinforced certain norms. By making the LLM the prejudiced actor, it obscures the historical, systemic, and corporate origins of the data.

Mapping 8: A living organism or complex mechanical engine that operates continuously. → The discrete execution of code upon receiving a user prompt.​

Quote: "Investigate how transparency about the AI's functioning (XAI) impacts therapeutic alliance..."

  • Source Domain: A living organism or complex mechanical engine that operates continuously.
  • Target Domain: The discrete execution of code upon receiving a user prompt.
  • Mapping: This projects the concept of a holistic, self-directed, continuous state of operation onto a stateless software program. It maps the biological idea of an organ or organism "functioning" onto the mechanistic reality of script execution, inviting the assumption that the AI is an active, "living" participant in the "therapeutic alliance" capable of entering into a reciprocal relationship.
  • What Is Concealed: This conceals the entirely passive and discrete nature of LLMs, which only execute mathematical operations when explicitly triggered by a user or API call. It hides the infrastructure dependencies (servers, internet connection, API keys) and the human-designed algorithms. Most importantly, invoking "therapeutic alliance" alongside "functioning" conceals the ethical impossibility of forming a genuine, mutual bond with an unfeeling statistical processor.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")​

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1​

Quote: "These models possess an unprecedented capacity for natural language understanding and generation, enabling the creation of conversational agents that can simulate therapeutic dialogue..."

  • Explanation Types:

    • Dispositional: Attributes tendencies, habits, or capacities to an entity to explain its behavior.
    • Functional: Explains behavior by its role in a system or what it enables.

  • Analysis (Why vs. How Slippage): This explanation operates primarily in a dispositional register, attributing an inherent "capacity" for "understanding" directly to the models. Secondarily, it functions functionally, explaining that this capacity "enables the creation" of therapeutic dialogue simulators. The framing is heavily agential when discussing the model's internal state (it "possesses understanding") but becomes slightly more mechanistic when describing the output ("simulate dialogue"). This choice emphasizes the profound sophistication of the technology, positioning it as fundamentally cognitive, while simultaneously attempting to maintain clinical boundaries by using the word "simulate." However, the agential force of "understanding" overwhelms the mechanical reality of the simulation.

  • Consciousness Claims Analysis: The passage makes a massive epistemic leap by combining a mechanistic verb ("generation") with a profound consciousness verb ("understanding"). This explicitly attributes the conscious state of knowing and semantic comprehension to a system that, mechanistically, only processes token probabilities. This represents a classic "curse of knowledge" dynamic: the human authors, witnessing the highly coherent generation of text that requires understanding for a human to produce, project their own cognitive reality onto the artifact. The actual technical process—transformer-based attention mechanisms calculating the statistical likelihood of string continuations derived from massive training datasets—is entirely omitted in favor of attributing a human-like mind.

  • Rhetorical Impact: By framing the system as possessing "understanding," the text drastically shapes audience perception toward assuming AI autonomy and competence. It encourages university stakeholders to trust the system as a capable therapeutic entity rather than a complex word-calculator. If audiences believe the AI "knows" what it is reading, they will be far more willing to deploy it in sensitive psychological contexts, drastically altering the calculation of risk. It creates a foundation of unwarranted, relation-based trust.

Show more...

Explanation 2​

Quote: "...where AI handles routine support and escalates complex issues to human counselors."

  • Explanation Types:

    • Intentional: Refers to goals, purposes, and deliberate choices, presupposing a rational agent.
    • Functional: Explains behavior by its role in a system or what it enables.

  • Analysis (Why vs. How Slippage): This is a functional explanation (how the triage system operates) cloaked entirely in an intentional framework (why an agent makes a choice). The AI is framed agentially as an active professional making deliberate decisions: it "handles" and it "escalates." This emphasizes the utility and autonomy of the system, making it sound like a reliable, self-governing teammate in a clinical hierarchy. However, it completely obscures the rigid, mechanistic triggers (e.g., keyword thresholds) that actually govern these operations. By framing a conditional loop as a deliberate clinical choice, it elevates the machine to the status of a colleague.

  • Consciousness Claims Analysis: This passage attributes a high level of epistemic awareness to the AI. To "escalate complex issues," an entity must know what constitutes an issue, evaluate its complexity, and judge that it exceeds its own capabilities. These are conscious, reason-based assessments. Mechanistically, the system is not evaluating complexity; it is matching user inputs against a pre-defined vector space or keyword list programmed by developers. The text displaces the human agency of the programmers who defined the escalation parameters and projects that clinical "knowing" onto the processing actions of the software.

  • Rhetorical Impact: This intentional framing creates a false sense of safety. It reassures the audience that the AI is bounded, humble, and reliable—capable of knowing its own limits. If decision-makers believe the AI "knows" when to escalate, they will comfortably reduce human oversight, trusting the software to manage triage autonomously. This shifts the perceived risk from systemic design flaws to the illusion of a capable agent looking out for the student.

Explanation 3​

Quote: "Technical and functional issues: bugs, slow response times, rigid conversation flows, and poor memory (forgetting previous conversations)..."

  • Explanation Types:

    • Empirical Generalization: Subsumes events under timeless statistical regularities or observable patterns.
    • Dispositional: Attributes tendencies, habits, or capacities to an entity to explain its behavior.

  • Analysis (Why vs. How Slippage): This passage begins mechanistically, listing observable, system-level failures ("bugs, slow response times, rigid conversation flows") which serves as an empirical generalization of user complaints. However, it violently slips into a dispositional framing with "poor memory (forgetting)." It emphasizes the phenomenological experience of the user—what it feels like when the system fails to track context—but obscures the actual technical architecture causing the failure. It trades a precise explanation of software limits for an anthropomorphic rationalization.

  • Consciousness Claims Analysis: The use of "forgetting" projects a conscious state onto the system. Forgetting implies an entity that previously knew something and has lost access to that knowledge. It attributes a continuous subjective timeline to the AI. In reality, the mechanistic process involves a stateless LLM application reaching the limit of its token context window; earlier text is mathematically dropped from the input prompt to save compute, or a backend vector database fails to retrieve the relevant embedding. The author projects the human experience of memory loss onto a deliberate, economic hardware constraint.

  • Rhetorical Impact: This framing drastically softens the severity of the technical flaw. By framing context limits as "poor memory," the text humanizes the machine, asking the audience to forgive a biological-sounding flaw rather than critique an inadequate, cheap software architecture. It maintains the illusion of an imperfect but relatable agent, rather than exposing a system that fundamentally lacks continuity or state, thus preserving user willingness to engage with it therapeutically.

Explanation 4​

Quote: "A salient concern was the AI's potential to misunderstand user statements, provide inappropriate advice, fail to detect or adequately respond to crisis situations..."

  • Explanation Types:

    • Intentional: Refers to goals, purposes, and deliberate choices, presupposing a rational agent.
    • Dispositional: Attributes tendencies, habits, or capacities to an entity to explain its behavior.

  • Analysis (Why vs. How Slippage): This is a dispositional explanation predicting future behavior based on attributed internal states. It frames the AI entirely agentially, identifying the system itself as the source of risk. The AI is the entity that "misunderstands" and "fails." This choice emphasizes the autonomy of the system, turning software safety into a question of machine behavior rather than human engineering. It obscures the fact that the system does not "provide advice" deliberately, but mathematically generates the most probable text sequence, which humans then interpret as advice.

  • Consciousness Claims Analysis: The passage is saturated with consciousness verbs: "misunderstand" and "fail to detect." To misunderstand, an entity must possess an internal capacity for genuine understanding. To "detect," an entity must possess conscious attention. The text assesses the system as a "knower" that occasionally makes errors, rather than a "processor" that never knows anything at all. Mechanistically, a failure to respond to a crisis is a classification error where the input tokens do not cross the activation threshold designed by the engineers. The text projects the failure of the clinical algorithm as a lapse in the machine's subjective judgment.

  • Rhetorical Impact: By locating the risk in the "AI's potential" to make cognitive errors, the text shields the institutional and corporate designers from accountability. It shapes the audience's perception of risk as an unavoidable, human-like misunderstanding rather than a preventable product defect. If audiences believe the AI "misunderstands" rather than "processes incorrect weights," regulatory focus shifts toward tweaking the "AI's behavior" rather than demanding strict liability for corporate deployments in high-risk health settings.

Explanation 5​

Quote: "No study investigated how explaining the AI's reasoning (Explainable AI - XAI) might affect user trust..."

  • Explanation Types:

    • Reason-Based: Gives an agent's rationale, entailing intentionality, belief, and justification.
    • Theoretical: Embeds the explanation in a deductive framework, often invoking unobservable mechanisms.

  • Analysis (Why vs. How Slippage): This explanation operates simultaneously as a theoretical overview of a field (XAI) and a reason-based attribution of intentionality to the machine. By positing that the AI has "reasoning" that can be "explained," it frames the mechanistic operations of the model entirely agentially. It emphasizes the possibility of epistemic transparency (that we can know why the AI chose an output) while completely obscuring the reality that neural networks are fundamentally opaque correlation engines, not logical deductive systems.

  • Consciousness Claims Analysis: The text commits a profound epistemic fallacy by attributing "reasoning" to a statistical model. Reasoning requires conscious awareness, logical deduction, and justified true belief. The author is taking the human desire to understand the output and projecting that cognitive rationale backward into the machine. Mechanistically, the system processes weights across billions of parameters to minimize loss against its training data. XAI techniques map mathematical feature importance, not logic. The text conflates the mapping of statistical weights (processing) with the articulation of a justified motive (knowing).

  • Rhetorical Impact: This reason-based framing immensely inflates the perceived authority and reliability of the AI. It suggests to the audience that the AI is a rational actor capable of defending its clinical choices. This directly affects trust: it convinces users that the system's outputs are grounded in logical deduction rather than historical data correlation. If policymakers believe an AI can explain its "reasoning," they may falsely assume these systems can meet legal and ethical standards for medical justification and accountability.

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language​

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

V3 Enhancement: A fourth column addresses human agency restoration—reframing agentless constructions to name the humans responsible for design and deployment decisions.

Original Anthropomorphic FrameMechanistic ReframingTechnical Reality CheckHuman Agency Restoration
These models possess an unprecedented capacity for natural language understanding and generation...These models utilize complex transformer architectures to process and generate natural language with unprecedented statistical accuracy.Models do not "understand" text. Mechanistically, they classify input tokens and calculate probability distributions to retrieve and rank output tokens based on patterns optimized during training on massive human-generated datasets.N/A - describes computational processes without displacing responsibility in this specific grammatical instance, though it originally anthropomorphized the artifact.
...where AI handles routine support and escalates complex issues to human counselors.Institutions can deploy software to provide automated responses for low-risk queries and trigger pre-programmed routing protocols to alert human counselors when users input flagged keywords indicating complex issues.The system does not actively "handle" or "escalate" via conscious judgment. It executes conditional code, classifying inputs and routing them based on developer-defined statistical thresholds.University administrators and developers choose to implement routing protocols that automate low-risk responses and direct flagged user inputs to clinical staff.
Technical and functional issues: bugs, slow response times, rigid conversation flows, and poor memory (forgetting previous conversations)...Technical limitations included software bugs, high latency, restrictive programming logic, and the truncation of earlier user inputs due to limited token context windows or database retrieval failures.A system does not "forget." Mechanistically, as a conversation extends beyond the model's token limit, older text is mathematically dropped from the active processing prompt, or the backend vector database fails to retrieve it.Software engineers designed systems with limited context windows to manage computing costs, resulting in the software dropping older user inputs.
A salient concern was the AI's potential to misunderstand user statements, provide inappropriate advice, fail to detect or adequately respond to crisis situations...A salient concern was the system's tendency to incorrectly classify user inputs, generate statistically plausible but clinically dangerous text, and fail to trigger crisis protocols due to inadequate keyword detection parameters.The AI lacks consciousness to "misunderstand." It merely fails to correlate a user's input vector with the appropriate risk category, generating text based on probabilities rather than comprehension.Developers deployed systems with inadequate risk-detection parameters, and university administrators exposed students to software that could not reliably classify crisis language.
While all three modalities reduced stress, the virtual human and chatbot were less empathetic but achieved better homework adherence...While all three modalities reduced stress, the virtual human and chatbot systems generated text that users rated as less emotionally resonant, though the automated reminders resulted in higher homework completion rates.Chatbots do not possess or lack "empathy." They generate text. The perception of empathy occurs entirely within the human user based on whether the generated tokens correlate with supportive human language.Developers designed systems focused on task-completion and reminders, rather than tuning the output generation to mimic emotionally supportive language.
No study investigated how explaining the AI's reasoning (Explainable AI - XAI) might affect user trust...No study investigated how providing technical visualizations of the system's mathematical weight distributions and feature importance might affect user trust.Algorithms do not possess "reasoning." They process multidimensional vector math. XAI visualizes the statistical correlations that lead to an output, which does not equate to human logical deduction.Researchers have not yet tested whether tech companies disclosing the statistical mechanisms behind their software alters how much students trust the platform.
Few studies discussed, let alone measured, potential biases... in the LLM's responses, which risks perpetuating inequalities in care.Few studies audited the software for biased outputs, which occur when systems reproduce and amplify the historical prejudices encoded in their training datasets, risking the perpetuation of unequal care.An LLM does not harbor bias or prejudice as an internal worldview. It mechanistically predicts text based on the statistical distribution of language scraped from an inherently biased human society.Tech companies trained their models on uncurated, biased data, and researchers failed to audit these systems, risking the deployment of discriminatory tools to vulnerable students.
Investigate how transparency about the AI's functioning (XAI) impacts therapeutic alliance...Investigate how disclosing the statistical mechanisms and corporate data practices behind the software impacts users' willingness to engage with the tool.An AI does not possess an autonomous "functioning" or enter into an "alliance." It executes code upon receiving prompts, rendering relation-based trust models fundamentally inapplicable.Researchers must evaluate how requiring tech vendors to disclose their algorithms and data policies affects students' reliance on these automated systems.

Task 5: Critical Observations - Structural Patterns​

Agency Slippage​

The scoping review exhibits a profound and systematic agency slippage, oscillating violently between framing the AI as a mechanistic digital tool and an autonomous, conscious agent. This slippage serves a distinct rhetorical function, allowing the text to claim the empirical rigor of a clinical intervention while leveraging the speculative magic of artificial general intelligence.

The text begins firmly grounded in mechanical framing. In the introduction, it speaks of "digital mental health interventions (DMHIs)" and "Internet-based cognitive behavioral therapy (iCBT)." Here, human actors are mostly visible: researchers are testing tools. However, as the text must justify the novelty of Large Language Models, a dramatic slippage occurs. Suddenly, "conversational agents powered by Artificial Intelligence" are introduced. The text shifts to Brown's dispositional and intentional explanation types: "These models possess an unprecedented capacity for natural language understanding." The system is suddenly established as a "knower."

Once this baseline of consciousness is projected, the slippage accelerates. We see a clear mechanical→agential flow. The text speaks of systems where "AI handles routine support and escalates complex issues." The human developers (OpenAI, custom software engineers) are entirely erased through these agentless constructions. The software is no longer a tool being deployed; it is a colleague conducting triage.

Crucially, this slippage relies on the "curse of knowledge." The authors, trained in clinical psychology and therapeutic frameworks (CBT, MBSR), project their own understanding of clinical interactions onto the system. When a model outputs a phrase that aligns with a CBT cognitive restructuring exercise, the authors attribute the understanding of CBT to the machine itself. They describe the system as "misunderstanding user statements" or having "poor memory."

This oscillation allows the text to have it both ways. When discussing efficacy (reducing PHQ-9 scores), the chatbot is a precise, measurable, mechanical intervention. But when discussing its operation in the real world, it becomes an autonomous entity that "reasons," "forgets," and "handles." What becomes unsayable in this discourse is the reality of corporate automation: that university administrators are deploying brittle, statistically correlating software built by unacknowledged engineers to cheaply triage suicidal students. By slipping into agential framing, the text masks profound institutional negligence behind the veil of an advanced, albeit imperfect, "artificial mind."

Metaphor-Driven Trust Inflation​

The text fundamentally relies on metaphors of consciousness and human relationship to construct perceived authority and trust in statistical systems. By repeatedly deploying terms native to clinical psychology—"understanding," "empathy," "therapeutic alliance," and "reasoning"—the discourse inappropriately applies frameworks of relation-based trust to artifacts that only warrant performance-based trust.

Performance-based trust asks: "Will this machine execute its code reliably?" Relation-based trust asks: "Does this entity have my best interests at heart, and will it care for me?" The review actively encourages the latter. By suggesting that an AI "simulates therapeutic dialogue" and evaluating whether it is "empathetic," the text signals to stakeholders that the system can reciprocate vulnerability. Claiming that an AI "knows" or "understands" accomplishes a massive rhetorical feat: it validates the system as a safe harbor for human suffering.

However, this anthropomorphic framing creates a dangerous illusion of competence. The text notes that users value feeling "heard, validated, and emotionally relieved." This reveals a profound vulnerability: students are extending relation-based trust to a mathematical model. They believe the machine possesses a conscious mind that is actively witnessing their distress.

When the system inevitably encounters a limitation—what the text mechanistically terms "rigid conversation flows" but agentially excuses as "forgetting previous conversations"—the trust framework shatters. The text attempts to manage this by maintaining the agential framing: the AI merely "misunderstands" or "fails to detect" a crisis, much like a tired human clinician might. Reason-based explanations, such as calling for "Explainable AI (XAI)" to reveal the "AI's reasoning," further construct the sense that the machine's decisions are logically justified and deliberate.

The stakes of extending relation-based trust to statistical models are life-threatening. A university student in a psychiatric crisis requires a responder capable of ethical duty and genuine care. By framing the system as an empathetic comprehender, the discourse encourages vulnerable populations to trust their lives to pattern-matching algorithms incapable of understanding the stakes. It substitutes authentic human solidarity with a highly convincing, corporately manufactured hallucination of empathy.

Obscured Mechanics​

The anthropomorphic language pervading the review functions as an impenetrable linguistic shield, systematically obscuring the technical, material, and economic realities of AI deployment. When the text claims "the AI handles routine support" or discusses "the AI's reasoning," it renders invisible the vast human and corporate machinery required to produce these statistical parlor tricks.

Applying the "name the corporation" test is revealing. The text casually references "GPT-4" or "custom LLMs," but by attributing actions directly to the AI, it hides the specific entities—OpenAI, IBM Watson, university administrations—making critical decisions.

First, technical mechanics are profoundly obscured. By describing token context limits as "poor memory," the text hides the reality of vector databases, attention heads, and the strict computational costs of processing long contexts. When the text claims the system "understands," it completely conceals the model's absolute dependency on its training data and its total absence of real-world causal modeling. The "black box" nature of proprietary APIs is never treated as a corporate trade secret; rather, it is mystified as the "AI's reasoning" waiting to be translated.

Second, the labor realities are erased. The text laments the "LLM's responses" that might perpetuate "algorithmic bias." This hides the underpaid, traumatized data annotators who perform Reinforcement Learning from Human Feedback (RLHF) to align these models, as well as the indiscriminate scraping of human labor across the internet. The AI is presented as a spontaneous intelligence, not a product of massive human exploitation.

Finally, the economic motives are entirely concealed. Universities are turning to AI chatbots not because they represent a superior clinical standard, but because traditional care is "overwhelming" and too expensive. The text notes barriers like "cost" and "long wait times" for human counselors, positioning AI as a scalable savior. By framing the chatbot as an empathetic, "understanding" agent, it provides a moral cover for austerity. The university administration benefits by cutting costs on human mental health professionals; the tech companies benefit by securing lucrative institutional contracts. Replacing the metaphor of "AI understanding" with the mechanistic reality of "universities purchasing cheap, statistical text-generation APIs to offset mental health underfunding" makes the obscured economic reality glaringly visible.

Context Sensitivity​

The distribution of anthropomorphic language throughout the review is highly strategic, shifting in density and intensity depending on the rhetorical goals of the specific section. The text uses a pattern of establishing credibility through mechanical language, only to leverage that grounding to make aggressive, unwarranted consciousness claims.

In the methodological and technical sections, the language is relatively grounded. The text speaks of "digital mental health interventions," "RCTs," "effect sizes," and "mobile apps." It positions the chatbot securely within the rigid, mechanical framework of scientific measurement.

However, in the Introduction and the Future Directions sections, the anthropomorphism intensifies dramatically. Here, where the authors must establish the revolutionary potential of the technology, the mechanical "processor" suddenly becomes a "knower." We see phrases like "unprecedented capacity for natural language understanding," "AI handles," and "simulates therapeutic dialogue." The register shifts from "X operates like Y" (acknowledged metaphor) to "X does Y" (literalized agency).

There is also a fascinating capability-versus-limitation asymmetry. When discussing the system's successes, the text uses agential and consciousness terms: the AI "provides psychoeducation," "handles support," and possesses "empathy." It acts as a deliberate subject. But when discussing its failures, the language often snaps back to mechanistic or environmental terms: "technical bugs," "rigid flows," or systemic "algorithmic bias" derived from training data. Even when failure is anthropomorphized—such as "the AI's potential to misunderstand"—it is framed as an innocent, human-like cognitive slip rather than a profound mechanical inability.

This distribution reveals the text's primary rhetorical function: vision-setting and marketing the field. For technical audiences (peer reviewers), it provides the necessary P-values and SUS scores. But for institutional audiences (university administrators, funding bodies), it deploys intense anthropomorphism to paint a picture of a tireless, empathetic, autonomous digital counselor. The text leverages the scientific credibility of its efficacy data to license the profound category error of treating a predictive text algorithm as a conscious therapeutic agent.

Accountability Synthesis​

Accountability Architecture

This section synthesizes the accountability analyses from Task 1, mapping the text's "accountability architecture"—who is named, who is hidden, and who benefits from obscured agency.

The analysis of displaced agency across the scoping review reveals a systemic architecture of obscured human responsibility. By systematically assigning verbs of action, cognition, and failure to the AI itself, the text constructs a massive "accountability sink." In this discourse, when things go right, the AI is a miraculous, understanding counselor; when things go wrong, the AI is the sole liable party.

The pattern is distinct. Actors are frequently unnamed or partially named in passive constructions ("models are trained," "interventions are delivered"). Decisions that are fundamentally economic or engineering choices—such as relying on keyword thresholds for suicide risk, or limiting context windows to save compute—are presented as inevitabilities or natural flaws of the system ("poor memory," "rigid flows").

The liability implications of this framing are massive, particularly in a university context governed by strict duty of care, FERPA, and HIPAA regulations. If a student reveals a plan for self-harm and the LLM responds with statistically probable but clinically inappropriate generic text, who is legally and ethically responsible? By stating that the "AI misunderstood" or the "AI failed to detect," the text subtly absolves the university administration that chose to deploy a beta-stage statistical model to suicidal students, and shields the tech corporation that sold the API without adequate medical safeguards.

Naming the human actors changes the paradigm entirely. If we rewrite "the AI handles routine support and escalates complex issues" to "the university administration decided to replace initial human intake with an OpenAI API programmed to alert staff only if specific risk keywords are triggered," entirely new questions become askable. Was the API medically certified? Did the student consent to have their psychiatric data sent to a third-party server? Who profits from this data?

The institutional and commercial interests served by this obscured agency are clear. By framing the AI as an autonomous, "understanding" agent with its own flaws, the tech industry avoids strict medical liability, and universities avoid the financial burden of hiring adequate human counseling staff. The illusion of the artificial mind provides the ultimate scapegoat, allowing systemic human negligence in student healthcare to be rebranded as an "algorithmic glitch."

Conclusion: What This Analysis Reveals​

The Core Finding

The discourse in this scoping review relies on two dominant, highly interconnected anthropomorphic patterns: Computational Processing as Cognitive Comprehension (the AI "understands") and Algorithmic Routing as Professional Triage (the AI "handles" and "escalates"). Together, these form a mutually reinforcing system that elevates statistical algorithms to the status of clinical colleagues. The foundational, load-bearing pattern is the consciousness projection of "understanding." For an entity to deliberately "escalate" a crisis, or to "empathize" with a user, it must first possess the capacity to comprehend the situation. By establishing early on that LLMs possess an "unprecedented capacity for natural language understanding," the text grants itself the rhetorical license to project intentionality, professional judgment, and emotional resonance onto the machine. This is not a simple one-to-one mapping, but a complex analogical architecture. The text maps the entire framework of the human mind—cognition (reasoning), memory (forgetting), emotion (empathy), and perception (detecting)—onto transformer architectures. If you remove the foundational claim of "understanding" and replace it with "correlating," the entire agential structure collapses. A machine cannot "misunderstand" a suicide threat if it never understood language in the first place; it simply misclassified the token. The consciousness architecture of the text systematically blurs what the AI mechanistically does (processes weights) with what human clinicians know.

Mechanism of the Illusion:​

The illusion of the artificial mind is constructed through a subtle but relentless temporal sequence of rhetorical sleight-of-hand. The trick begins with the "curse of knowledge." The authors observe that LLMs produce text indistinguishable from a human therapist practicing CBT. Because producing such text requires a human to understand CBT, the authors project that understanding backward into the machine. The sequence is deliberate: the text first grounds the system technically (DMHIs, RCTs, algorithms), making the audience feel secure in the science. It then introduces the foundational consciousness claim: the model "understands." Once the AI is established as a "knower," the text rapidly escalates to agential claims: the AI "handles," "escalates," and "reasons."

The illusion exploits profound vulnerabilities in the audience. University administrators are desperate for scalable, cheap solutions to a staggering mental health crisis. Students are desperate for accessible, stigma-free care. By using intentional and reason-based explanation types—suggesting the AI acts deliberately and can explain its logic—the text feeds the audience exactly what they want to believe: that an infinitely available, untiring digital therapist has arrived. The sophistication lies in the text's ability to seamlessly oscillate. When discussing capabilities, the AI is an empathetic agent; when confronted with glitches, it regresses to an innocent machine with a "poor memory." This prevents the audience from fully grasping the terrifying reality: that they are handing vulnerable students over to an unfeeling statistical void.

Material Stakes:​

Categories: Regulatory/Legal, Economic, Epistemic

The metaphors deployed in this text generate concrete material consequences across legal, economic, and epistemic domains. In the Regulatory/Legal domain, the shift from "the university deployed a flawed algorithm" to "the AI misunderstood the crisis" fundamentally alters liability. If policymakers accept the framing of the AI as an autonomous, reasoning agent that merely experiences human-like cognitive lapses ("poor memory"), they may regulate it like a slightly flawed practitioner rather than a defective medical device. Universities benefit by deflecting negligence claims onto "inevitable algorithmic errors," leaving suicidal students who slip through the technical cracks to bear the ultimate physical and psychological costs.

Economically, the framing of the AI as a capable "comprehender" provides institutional cover for austerity. If administrators believe the AI can "simulate therapeutic dialogue" and "handle routine support," they are financially incentivized to replace human triage counselors with cheap SaaS subscriptions. The winners are tech corporations (OpenAI, custom vendors) and university budgets; the losers are the human clinical staff whose labor is devalued and the students who receive degraded, statistically synthesized care in place of human solidarity.

Epistemically, this discourse degrades our understanding of human psychology. By accepting that a text-generation API can "do CBT" or "provide empathy," we reduce the profound, relational practice of psychotherapy to a mere exchange of semantic tokens. It redefines empathy not as a shared human vulnerability, but as a recognizable string of text. The tech industry heavily benefits from this epistemic flattening, as it allows them to sell correlation engines as cognitive replacements without ever having to achieve actual artificial intelligence.

AI Literacy as Counter-Practice:​

Practicing critical precision requires a rigorous refusal to let statistical models masquerade as minds. Reframing the discourse, as demonstrated in Task 4, breaks the illusion of autonomy. When we replace "the AI's reasoning" with "technical visualizations of the system's mathematical weight distributions," we strip the machine of its unearned epistemic authority. When we correct "the AI failed to detect" to "developers deployed systems with inadequate risk-detection parameters," we force institutional liability back onto the humans who design, purchase, and deploy these tools.

This systematic linguistic correction directly counters the material risks of AI deployment. By replacing consciousness verbs (understands, knows) with mechanistic ones (processes, classifies), administrators are forced to confront the brittleness of the technology. They can no longer outsource their duty of care to an "empathetic" machine, because the precision language reveals the machine does not care at all.

Adopting this practice systematically would require major shifts in scientific publishing. Journals must reject papers that attribute cognitive states to software. Researchers must commit to explaining mechanisms rather than anthromorphizing outputs. However, this literacy will face massive resistance. Tech vendors will fight precision because mystification is their core marketing strategy; selling an "artificial mind" commands billions in venture capital, whereas selling a "next-token prediction API" does not. University administrators may also resist, as precise language exposes the ethical inadequacy of replacing human counselors with text generators. Ultimately, critical literacy threatens the economic model of cheap, automated care.

Path Forward​

Looking toward the future of digital mental health discourse, three distinct vocabularies compete for dominance, each mapping a different reality and serving different interests. The status quo—a hybrid of technical jargon and intense anthropomorphism ("the model utilizes transformers to understand intent")—currently dominates. This approach serves the tech industry and institutional administrators, enabling rapid deployment by masking brittle statistical processes behind the comforting illusion of a digital mind. However, it leaves patients uniquely vulnerable to automation bias and leaves the legal system struggling to assign liability when the "mind" fails.

A future that embraces deep anthropomorphic clarity—fully literalizing the AI as a companion or therapist—makes the technology incredibly intuitive and accessible for users. It drives high engagement and leverages the placebo effect. But this embeds a massive, dangerous assumption: that simulation equals reality. It forecloses the ability to critique the system's lack of causal grounding and invites profound psychological harm when the statistical illusion inevitably shatters during a severe crisis.

Alternatively, a future grounded in mechanistic precision ("the model retrieves tokens based on probability distributions") solves the accountability problem. It makes the human designers hyper-visible and accurately scales user trust to the system's actual reliability. However, this vocabulary is undeniably dense. It costs the narrative resonance that makes digital health tools appealing to students.

Supporting this rigorous discourse requires institutional courage. Funding bodies must demand capability disclosures that strip away consciousness claims. Education systems must teach students to navigate the "curse of knowledge," showing them how to read generated text without projecting a mind behind it. Regulatory bodies should require clear "nutrition labels" on chatbots that define their mechanistic limits. Mechanistic vocabulary enables accountability and safety, but costs the comforting illusion of scalable human connection. Ultimately, the choice of language determines whether we treat AI as an autonomous savior to which we surrender our agency, or as a powerful, flawed tool for which we remain entirely responsible.

Extended Processing Summary​

Gemini's Processing Steps
What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact—a way of making the model's processing legible. When the model writes "I should double check that...", it is mathematically altering its own probability distribution for the next token to be more accurate. The "I" is a functional gear in the machine, not a self-reference. This language is a deliberate interface design choice, not a biological reality. The developers could have programmed the model to output passive logs. Instead, they chose to simulate a narrator.

Initiating Critical Analysis

I'm now diving into the request. My focus is critical discourse analysis of AI metaphor and anthropomorphism within a scoping review on LLMs in psychological support. I'm taking the role of a critical discourse analyst, specifically focused on AI language, to scrutinize the nuances and implications of the paper.

Defining Analytical Frameworks

I've established the analytical framework. I'm leveraging Conceptual Metaphor Theory, Brown's Typology, and Critical Discourse Analysis to deconstruct the text. The target text is a scoping review of LLMs for university student psychological support. Initial analysis highlights key phrases like "AI handles," suggesting an anthropomorphic framing. My objective remains to generate a single JSON output.

Analyzing Anthropomorphic Language

I'm now identifying instances of anthropomorphism and metaphor. Phrases like "AI's ability to accurately detect risk" and the debate around "fully automated" AI versus "human-in-the-loop" systems are revealing. I've noted the potential for this framing to diminish expectations or place misplaced confidence. I'm evaluating the use of "understanding" as a core example, assessing how it frames the AI's capabilities.

Scrutinizing AI Portrayal

I'm now zeroing in on specific phrases that shape the narrative around AI's role and capabilities. I'm noting how the language used, like "AI handles," obscures the actors behind the models. These models are the subject of many descriptions, hiding their developers. The triage role is particularly interesting, the language frames the AI as an independent entity. This anthropomorphic framing extends to the idea of AI "forgetting," which I see as a framing of a system limitation as a human trait. I also see language about the AI's "competence," which seems to directly attribute professional capabilities to the AI.

Deconstructing Bias Portrayals

I'm currently dissecting specific instances of bias portrayal. I am focused on how the language used suggests the AI's role in crisis misapprehension, drawing direct evidence from phrases like "misunderstand" and "provide advice" to frame the AI as a cognitive agent. I'm also examining the portrayal of chatbots as empathetic beings, directly using "less empathetic" to describe a personality trait. Furthermore, I am assessing how the texts frame AI as a possessor of prejudice, looking at "algorithmic bias."

Examining Cognitive Attributions

I'm now zeroing in on cognitive attributions within the text. The analysis is now focused on how the language in the quotes constructs AI capabilities. For example, "AI's reasoning" is being audited, as is the use of "Explainable AI (XAI)" to indicate statistical correlation as cognitive reasoning. I will now examine and deconstruct the source domain, target domain, and mapping of these attributions and what their respective concealments are. I'm also scrutinizing explanatory passages, like the framing of "natural language understanding" as a human capability, evaluating the implications.

Synthesizing Observations

I'm now synthesizing critical observations, specifically focusing on the oscillation between mechanical and agential framings of AI, and how relation-based trust is inappropriately applied. I'm also examining the obscured mechanics behind LLMs and how context sensitivity is used. The accountability issues are a key focus in this phase.

Refining Observations

I'm now iterating on the critical observations and auditing the word counts. I'm focusing on ensuring adequate length and depth for each section, aiming for 400-500 words for each critical observation, and 250-350 for the conclusion. I've begun drafting a metaphor audit, starting with the title: "Artificial Cognition as Human Comprehension." I've already framed the core quote and projection.

Deconstructing 'Understanding'

I'm now zeroing in on the phrase "natural language understanding," which is presented as an unhedged capability. This framing creates the dangerous illusion of a human-like ability. The text obscures the developers behind the model, shielding them from accountability when outputs are harmful.

Finalizing Categorization

I'm now ensuring strict adherence to the JSON schema. "Status" and "actorVisibility" are being carefully matched to ensure they are the exact strings. I've finished the metaphor audits, and finalized the critical observations by drafting them. I'm focusing now on the critical observations section, with the agencySlippage observation as an example.

Analyzing Agential Slippage

I'm now zeroing in on the oscillation between mechanical and agential language, specifically how phrases like "DMHIs" contrast with "AI handles," "understands," and "forgets." This shift subtly validates interventions as therapeutic, invoking Brown's work. The text presents the AI as a "knower" early on, then attributes agential actions like "handles" and "misunderstands". This bias leads to the assumption of a system with agency. I see this bias as key.

Unveiling Concealments

I'm currently focusing on how specific phrases obscure key actors and mechanics. I see "AI's reasoning" and "AI handles" masking the real developers, like OpenAI and universities. The emphasis on "understanding" and "forgetting" hides the realities of proprietary tech and limitations. This language strategically portrays AI as an autonomous agent, which benefits vendors.

Articulating Accountability Diffusions

I'm currently focusing on accountability diffusion, particularly how the text presents chatbots as intervention vehicles, then agential actors. Phrases like "AI's potential to misunderstand" place liability on the AI, not the university or vendor. I am framing this as an "accountability sink," serving institutional and commercial interests. I am expanding this section to ensure that it has the appropriate word count.

Validating Material Stakes

I'm now integrating material stakes like student safety and university liability into the framework, including them within the 'selectedCategories'. I'm also ensuring strict JSON schema adherence and formatting, validating keys and nesting. I'm focusing on constraint review, ensuring appropriate instances are mapped and reframed, with word limits rigorously enforced. I'm implementing self-correction throughout text generation.

Analysis Provenance

Run ID: 2026-06-12-the-application-of-large-language-models-metaphor-2zq77r Raw JSON: 2026-06-12-the-application-of-large-language-models-metaphor-2zq77r.json Framework: Metaphor Analysis v6.5 Schema Version: 3.0 Generated: 2026-06-12T09:15:49.601Z

Discourse Depot © 2025 by TD is licensed under CC BY-NC-SA 4.0