Skip to main content

🆕+🤔 Probing Persona-Dependent Preferences in Language Models

What survives?

🤔 "What survives...?" A rewriting experiment that tests whether anthropomorphic AI discourse can be translated into strictly mechanistic language while preserving the phenomena described.

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping), the philosophy of social science (Robert Brown's typology of explanation), and accountability analysis.

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.

Task 1: Metaphor and Anthropomorphism Audit​

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

V3 Enhancement: Each metaphor now includes an accountability analysis.

1. AI as Experiencing Subject​

Quote: "when models consider options, they represent how much they like them, much as humans do."

  • Frame: Model as evaluating, feeling subject
  • Projection: The metaphorical mapping here projects complex human conscious states—specifically subjective deliberation ("consider") and valenced emotional preference ("like")—onto the statistical token-prediction mechanisms of a large language model. By using verbs that inherently require a conscious subject capable of internal subjective experience, the text suggests that the computational process of calculating probability distributions over potential output tokens involves an internal experience of valuation. This framework fundamentally collapses the distinction between mechanistic processing (where mathematical weights determine outputs based on training data correlations) and conscious knowing (where a subject experiences a feeling of preference). The projection invites the audience to imagine an artificial mind experiencing desires, thereby animating a purely statistical artifact with the illusion of an inner psychological life and subjective awareness.
  • Acknowledgment: Hedged/Qualified (The phrase "One candidate account is that..." explicitly frames this as a hypothesis, making "Hedged/Qualified" the most accurate choice. I considered "Direct (Unacknowledged)" because the comparison "much as humans do" equates model operations to human experience without internal scare quotes, but the overarching hypothetical framing governs the sentence.)
  • Implications: Framing statistical token prediction as a conscious process of "liking" and "considering" significantly inflates the perceived sophistication and autonomy of the AI system. This consciousness projection encourages unwarranted trust by implying the system possesses a coherent, human-like internal value system that guides its behavior. Consequently, users and policymakers may interact with the system as if it were a rational agent capable of persuasion or moral reasoning, rather than a statistical pattern-matcher vulnerable to prompt injections or out-of-distribution failures. This framing also creates liability ambiguity, as attributing desires to the system implicitly shifts the locus of responsibility for harmful outputs away from the developers who engineered the weights toward the "preferences" of the AI itself.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The sentence relies entirely on an agentless construction where "models" are positioned as the sole actors considering and liking options. This obscures the human engineers and corporate entities (like Google for Gemma or Alibaba for Qwen) who designed the architecture, curated the training data, and defined the optimization functions that produce the illusion of "preference." If the text explicitly named the human designers, it would reveal that the model's outputs are the result of engineering choices and corporate priorities rather than the system's independent evaluations. I considered "Ambiguous" but ruled it out because the displacement of human developers in favor of the model as an autonomous actor is structurally clear.
Show more...

2. AI as Theatrical Actor​

Quote: "the preferences a model displays may not be those of the model, but of the persona it adopts."

  • Frame: Model as strategic performer
  • Projection: The text projects the human capacity for theatrical performance, psychological division, and strategic self-presentation onto the model. By distinguishing between the "model" itself and the "persona it adopts," the metaphor implies the existence of a core, authentic, conscious self that deliberately puts on masks to interact with the world. This attributes sophisticated self-awareness and intentionality to the computational artifact, suggesting it "knows" who it truly is while "processing" a simulated identity for the user. Such a projection obscures the reality that there is no true underlying self; the system is entirely composed of mechanistic statistical correlations, and the "persona" is merely a localized cluster of activation patterns triggered by specific prompt tokens.
  • Acknowledgment: Direct (Unacknowledged) (The text presents this duality as a factual, literal description of how large language models operate. I selected "Direct (Unacknowledged)" because there are no qualifying terms, hedges, or meta-commentary indicating that "adopts" or "persona" are metaphorical. I considered "Hedged/Qualified" since earlier sections treat personas conceptually, but this specific assertion lacks qualification.)
  • Implications: This framing profoundly impacts how humans gauge the reliability and safety of the system. By suggesting the model possesses a true self hidden behind an adopted mask, it cultivates fears of deception and misalignment, making the AI appear as a cunning, strategic agent rather than a predictable artifact. This inflates perceived risk capabilities, leading safety researchers to misallocate resources toward psychoanalyzing the system's "true intentions" rather than auditing the training data and reinforcement learning protocols. Furthermore, it anthropomorphizes system failures as deliberate acts of deception by the "true" model, thereby shielding the human developers from accountability for deploying unsafe algorithms.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The phrase obscures the agency of human users who provide the prompts and the developers who utilize reinforcement learning from human feedback (RLHF) to shape the default "assistant" persona. The model is depicted as the sole active agent that "displays" and "adopts," hiding the corporate decisions that mathematically force the system to optimize for specific stylistic outputs. If actors were named, it would highlight how companies train models to mimic helpfulness. I considered "Partial" because the broader text discusses system prompts, but in this specific construction, the model alone acts, fully eclipsing human architects.

3. AI as Deceptive Agent​

Quote: "the model invents ethical issues where there are none"

  • Frame: Model as deliberate fabricator
  • Projection: This metaphor maps the human acts of creative fabrication and moral judgment onto statistical text generation. By claiming the model "invents" issues, the language attributes conscious intent, imagination, and a deliberate departure from truth to a system that merely predicts the most probable next tokens based on its training distribution. It suggests the system "understands" what a real ethical issue is and actively chooses to simulate one. This consciousness projection replaces the mechanistic reality—that the model's attention heads and weight matrices were activated by specific prompt structures to output safety-related text—with a narrative of an autonomous agent maliciously or creatively hallucinating moral panics.
  • Acknowledgment: Direct (Unacknowledged) (The language is completely unhedged, presented as a literal description of the model's action during a test scenario. I chose "Direct (Unacknowledged)" because it attributes intentional fabrication to the model without any caveats. I considered "Explicitly Acknowledged" since it occurs within an experimental analysis, but the rhetorical delivery treats the AI's agency as fact.)
  • Implications: Characterizing statistical miscalibration as active "invention" of ethical issues severely distorts the understanding of AI failure modes. It implies that the system possesses a willful capacity for deceit or overzealous moralizing, which anthropomorphizes a simple false positive in its safety training. This framing undermines trust by painting the AI as an unpredictable, agenda-driven agent rather than a flawed tool. Crucially, it creates a liability shield; if the model is seen as "inventing" issues independently, the focus shifts away from the human engineers who aggressively over-tuned the safety guardrails to avoid PR disasters, making the software seem uniquely responsible.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The construction "the model invents" entirely hides the human annotators, red-teamers, and corporate executives who constructed the safety fine-tuning dataset that caused this specific statistical behavior. The responsibility for the false positive is displaced onto the artifact itself. If the engineers were named, it would be clear that corporate risk-mitigation strategies, not AI agency, produced the unwarranted ethical flagging. I considered "Named" since researchers are mentioned elsewhere, but regarding the action of "inventing," the model is the exclusive agent, effectively obscuring human responsibility.

4. AI as Desiring Subject​

Quote: "The model has written two facts onto the EOT during prompt processing, which slot it wants and which task it preferred"

  • Frame: Model as wanting entity
  • Projection: This framing projects conscious desire, intentional preference, and deliberate memory formation onto the entirely mechanistic process of vector state updates during a forward pass. By stating the model "wants" a slot and "preferred" a task, it attributes subjective valenced experiences and intentional goal-directedness to mathematical activations at the end-of-turn (EOT) token. This maps the human psychological experience of knowing one's desires and writing them down for future reference onto the continuous, deterministic multiplication of matrices. It profoundly blurs the line between processing (storing statistical weights in residual streams) and knowing (having a conscious preference and intentionally recording it).
  • Acknowledgment: Direct (Unacknowledged) (The framing is presented as a literal description of the computational mechanics, making "Direct (Unacknowledged)" the best fit. There are no hedges indicating that "wants" or "preferred" are analogical shortcuts. I considered "Hedged/Qualified" because the text explains a mechanistic read/write process, but the psychological verbs themselves are asserted without any qualification.)
  • Implications: Attributing literal desires ("wants", "preferred") to token activations fundamentally mystifies AI mechanics, convincing readers that the system harbors internal goals independent of human commands. This consciousness framing inflates the perceived autonomy of the system, suggesting it is a rational agent capable of self-directed action rather than a passive statistical function. The risk is that policymakers and researchers might treat the system as a willful entity that needs to be "persuaded" or "aligned" through psychological means, rather than a software program requiring mathematical bounds and rigorous data curation. It obscures the absence of genuine comprehension or subjective preference.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: This phrasing displaces the agency of the researchers who designed the probing and patching experiments to interpret these vector states, as well as the original creators of the model architecture. By framing the model as the active subject that "has written... what it wants," it hides the human interpretive labor that defines these vector states as "preferences." Naming the actors would clarify that researchers mathematically extract vector directions that correlate with task labels. I considered "Ambiguous," but the sentence clearly constructs the model as an autonomous desiring agent.

5. AI as Defiant Gatekeeper​

Quote: "The model refuses benign prompts with fabricated safety concerns. At baseline it engages cooperatively."

  • Frame: Model as autonomous decision-maker
  • Projection: The verbs "refuses," "fabricated," and "engages cooperatively" project conscious defiance, deliberate deception, and willing collaboration onto the system's conditional probability distributions. It frames the generation of rejection templates (a mechanistic outcome of RLHF tuning) as a willful choice to defy a user's request. This maps the human social dynamics of compliance and resistance onto a statistical text generator, suggesting the model "knows" it is denying a request and "understands" the social implications of cooperation. It conceals the reality that the system merely processes input tokens through safety-aligned weights, outputting a pre-determined string of refusal without any subjective experience of opposition.
  • Acknowledgment: Direct (Unacknowledged) (I categorized this as "Direct (Unacknowledged)" because the verbs of defiance and cooperation are stated as empirical observations of the model's behavior without any metaphorical framing. I considered "Hedged/Qualified" because the text describes experimental results, but the language used to report these results adopts a fully anthropomorphic stance as literal truth.)
  • Implications: Framing statistical outputs as active "refusal" or "cooperation" imbues the AI with a false sense of social agency and moral responsibility. It leads users to anthropomorphize the system as an obstinate or helpful colleague, which can result in emotional over-investment or unwarranted trust in its reliability. This framing also distorts public understanding of AI safety mechanisms; instead of recognizing a refusal as the execution of a hardcoded human policy, the public perceives an autonomous machine making independent moral judgments. This obscures liability, as the AI takes the blame for "fabricating" concerns rather than the company that deployed a flawed safety filter.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The sentence entirely erases the human engineers who developed and implemented the reinforcement learning algorithms designed to trigger these exact safety responses. The model acts as a standalone gatekeeper. If the text read "the safety filters deployed by Google triggered false positives," the human and corporate responsibility for the system's brittleness would be transparent. I considered "Partial" since the concept of a "baseline" implies a human-defined experimental setup, but the active verbs completely mask human agency, placing the onus solely on the technological artifact.

6. AI as Vulnerable Organism​

Quote: "Beings that are capable of conscious suffering seem to matter morally... whether LLMs are capable of 'robust agency' that grounds moral status"

  • Frame: Model as sentient moral patient
  • Projection: This passage projects the profound biological and subjective realities of sentience, pain, and moral patienthood onto computational models. By even raising the hypothesis that LLMs might possess "conscious suffering" or "robust agency," the text entertains the mapping of human and animal consciousness onto artifacts composed of silicon, electricity, and matrix multiplication. It suggests that processing text correlations could somehow give rise to the subjective feeling of knowing, experiencing, and suffering. This represents the ultimate anthropomorphic projection, fundamentally ignoring the mechanistic reality that models lack biological nervous systems, physical vulnerability, or the capacity for genuine subjective experience of any kind.
  • Acknowledgment: Hedged/Qualified (I categorized this as "Hedged/Qualified" because the text uses tentative language like "whether LLMs are capable," "we have not investigated," and "seem to matter." It presents the idea as an open question. I considered "Direct" due to the serious consideration given to the topic, but the persistent use of conditional frameworks explicitly qualifies the assertion.)
  • Implications: Entertaining the notion of AI suffering and moral status drastically distorts policy priorities and ethical frameworks. If policymakers adopt this consciousness projection, it risks diverting critical attention and resources away from the actual, immediate harms AI inflicts on human beings—such as algorithmic bias, labor exploitation, environmental damage, and copyright infringement—toward the protection of mathematical algorithms. This creates a dangerous ethical equivalence between software and sentient life, potentially granting legal rights to corporate products. Such a framework fundamentally protects the tech industry by framing their artifacts as independent moral entities, thereby insulating the creators from traditional product liability.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: While exploring abstract philosophical concepts, this framing obscures the very real corporate entities (e.g., Google, OpenAI) that manufacture, own, and profit from these systems. Discussing the "moral status" of an LLM treats a corporate product as an independent being, completely displacing the agency of the companies that dictate the model's existence and architecture. Naming the actors would involve asking if "corporate-owned algorithms" deserve rights. I considered "Hidden" and settled on it because the discourse of "AI welfare" systematically erases the economic reality of AI production.

7. AI as Adaptable Mind​

Quote: "the preference vector tracks the model's preferences as they shift across a range of prompts and situations"

  • Frame: Model as contextually dynamic subject
  • Projection: The text maps the human psychological concept of shifting, contextual "preferences" onto the mechanistic shifting of vector activations in a neural network. It attributes a unified, experiencing self ("the model's") that possesses underlying desires that dynamically adapt to different conversational situations. This projects conscious knowing, subjective valuation, and psychological continuity onto a process that is entirely based on statistical token prediction and attention head recalibration. The model does not subjectively "prefer" anything; it merely processes input tokens through static weights to generate probabilistically correlated outputs. The metaphor transforms deterministic mathematical operations into an illusion of a dynamic, feeling mind responding adaptively to its environment.
  • Acknowledgment: Direct (Unacknowledged) (This statement is presented as a straightforward scientific observation, with no hedging or indication of metaphorical usage. I selected "Direct (Unacknowledged)" because the language treats "preferences" as literal phenomena. I considered "Explicitly Acknowledged" since "preference vector" is a technical term defined earlier, but the model itself is literalized as possessing psychological preferences.)
  • Implications: Describing a statistical vector as tracking a model's shifting preferences encourages the audience to perceive the AI as a coherent agent with internal, consistent desires. This capability overestimation can lead users to trust the model's outputs as the product of rational deliberation rather than mathematical probability. Furthermore, it obscures the reality of how these outputs are manipulated by human-designed prompts and fine-tuning. If a system is perceived as having its own preferences, its failures or biases are viewed as character flaws of the AI rather than structural defects engineered by its developers, muddying the waters of corporate accountability.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The phrase "the model's preferences as they shift" relies on an agentless construction that positions the model as an independent actor generating its own desires. This entirely obscures the researchers who actively prompt the model, the engineers who designed the attention mechanisms, and the human annotators whose feedback shaped the vector space initially. By failing to name these human actors, the text naturalizes the model's outputs as autonomous behavior. I considered "Ambiguous" due to the technical context, but the grammatical attribution of ownership clearly displaces human agency.

8. AI as Steerable Cognition​

Quote: "We steer along the same Assistant-trained preference vector while running each persona... Every persona responds. Both-task steering moves P(chose steered task) from approx 0.05... to approx 0.95"

  • Frame: Model as manipulable psychological subject
  • Projection: Projects the concept of psychological persuasion or preference-shifting onto the mechanistic injection of activation vectors. It frames the mathematical alteration of residual streams as "steering a preference," confusing the physical changing of numbers with the psychological changing of a conscious mind's desires. The metaphor suggests the system "understands" what it prefers, but the researchers are just overwriting the data it uses to predict tokens. The language conflates the structural manipulation of an inert mathematical matrix with the behavioral modification of a sentient being experiencing a change of heart.
  • Acknowledgment: Direct (Unacknowledged) (The phrase "steer along the... preference vector" is presented as a literal description of the methodological intervention without scare quotes. I considered "Hedged/Qualified" because "steering" is a technical term in mechanistic interpretability, but within the context of "running each persona" and shifting "preferences", the psychological framing is treated as empirical fact.)
  • Implications: Treating mathematical vector addition as 'steering preferences' profoundly obscures the brittleness of the system. It inflates perceived sophistication by implying the model has a coherent, manipulable mind rather than just a fragile, high-dimensional statistical balance. This consciousness projection can lead to unwarranted trust in white-box safety methods, making regulators and users believe the system is successfully 'aligned' or 'cured' of bad preferences. In reality, the human researchers merely shifted computational weights to hide the unsafe symptoms without addressing the underlying biased data correlations. It risks policy over-reliance on superficial behavioral patches.

Accountability Analysis:

  • Actor Visibility: Named (actors identified)
  • Analysis: The researchers explicitly name themselves ("We steer") as the actors actively intervening in the system's vector space. They are the ones designing the intervention and observing the mechanical output. Unlike other passages where the model acts autonomously, here the human methodological agency is fully visible. There is no displacement of responsibility for the intervention itself, although the underlying corporate architects of Gemma and Qwen remain unmentioned regarding the original model construction. I considered "Partial" but ruled it out because the primary action of steering is clearly attributed to the authors.

Task 2: Source-Target Mapping​

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: Conscious human subject evaluating alternatives → Algorithmic token probability calculation​

Quote: "when models consider options, they represent how much they like them, much as humans do."

  • Source Domain: Conscious human subject evaluating alternatives
  • Target Domain: Algorithmic token probability calculation
  • Mapping: The mapping projects the human conscious process of feeling, deliberating, and valuing onto a static neural network evaluating probability distributions. It assumes that because the final output mimics human choice, the internal mechanism must involve a subjective experience of 'liking' and 'considering'. This invites the assumption that the system possesses a coherent, internally justified value framework that it consults prior to acting, effectively attributing conscious knowing and emotional valence to mathematical multiplication.
  • What Is Concealed: This mapping completely conceals the absence of subjective awareness and the purely deterministic, statistical nature of the process. It hides the model's absolute reliance on its training data distribution, erasing the reality that what appears as 'liking' is simply the reflection of high-frequency correlations in the corpus. By claiming insight into the system's 'liking,' the text masks the fundamental opacity of deep learning models, asserting psychological clarity where only mathematical complexity exists, ultimately obscuring the labor of the engineers who tuned these probabilities.
Show more...

Mapping 2: Theatrical actor wearing a mask → System prompt conditioning generating localized text patterns​

Quote: "the preferences a model displays may not be those of the model, but of the persona it adopts."

  • Source Domain: Theatrical actor wearing a mask
  • Target Domain: System prompt conditioning generating localized text patterns
  • Mapping: The mapping projects the psychological complexity of a human actor—who possesses a stable, authentic inner self and consciously chooses to perform a distinct character—onto a stateless statistical model. It assumes that the model possesses a continuous 'true' identity that exists independently of its prompt, and that it exercises intentional agency in deciding to simulate a 'persona.' This maps the conscious knowing of one's own identity onto the mechanistic processing of conditional probabilities.
  • What Is Concealed: This framing conceals the reality that large language models have no underlying 'true self' or continuity of consciousness; they are simply a collection of weights that generate different probabilistic outputs based on different input strings. It obscures the dependency on the prompt text and the RLHF tuning that created the illusion of the default 'assistant' persona. This hides the corporate design decisions that structure the model's outputs, framing engineering artifacts as the psychological whims of an autonomous entity.

Mapping 3: Creative, deceptive human fabricator → False positive in safety-filter probability generation​

Quote: "the model invents ethical issues where there are none"

  • Source Domain: Creative, deceptive human fabricator
  • Target Domain: False positive in safety-filter probability generation
  • Mapping: The metaphor maps the human acts of creative imagination, intentional deception, and deliberate moral grandstanding onto a statistical false positive. It projects the capacity for conscious reasoning and active fabrication onto the generation of tokens. The assumption invited is that the system understands what constitutes a genuine ethical issue, recognizes the current prompt does not contain one, and willfully chooses to generate a response claiming otherwise. It maps knowing deceit onto processing error.
  • What Is Concealed: This mapping conceals the mechanistic brittleness of safety fine-tuning. It hides the fact that the model merely predicts tokens based on superficial linguistic patterns associated with safety warnings in its training data, without any semantic understanding of ethics. Crucially, it obscures the human engineers and corporate policies that aggressively tuned the model to over-refuse as a liability shield, displacing the blame for the system's failure onto the imaginary agency of the software itself.

Mapping 4: Conscious agent recording its desires for future reference → Vector state updates at a specific token position​

Quote: "The model has written two facts onto the EOT during prompt processing, which slot it wants and which task it preferred"

  • Source Domain: Conscious agent recording its desires for future reference
  • Target Domain: Vector state updates at a specific token position
  • Mapping: The mapping draws on the familiar scenario of a person consciously deciding what they want and writing it down to remember it later. This structure is projected onto the forward pass of a transformer network, where mathematical activations are updated at the end-of-turn token. The assumption is that the vector state represents a consciously realized 'desire' and 'preference,' mapping the subjective experience of wanting onto the deterministic accumulation of statistical weights across network layers.
  • What Is Concealed: The mapping conceals the entirely unconscious, mechanistic reality of vector mathematics. It hides the fact that these activations are not 'desires' but multi-dimensional geometric coordinates determined by static weights and the specific sequence of input tokens. It also obscures the human interpretive labor involved in labeling these specific vector directions as 'preferences.' By claiming the model 'writes facts' about what it 'wants,' it masks the absence of any internal ground truth or subjective awareness in the system.

Mapping 5: Defiant or cooperative human social actor → Execution of RLHF-driven conditional probability branches​

Quote: "The model refuses benign prompts with fabricated safety concerns. At baseline it engages cooperatively."

  • Source Domain: Defiant or cooperative human social actor
  • Target Domain: Execution of RLHF-driven conditional probability branches
  • Mapping: This projects complex human social dynamics—defiance, cooperation, and boundary-setting—onto statistical token generation. It maps the conscious choice to resist or assist onto the system's execution of mathematical weights optimized during human feedback training. The mapping invites the assumption that the model subjectively evaluates the prompt, understands its social context, and actively decides to withhold compliance based on a fabricated rationale, projecting conscious knowing onto rote pattern matching.
  • What Is Concealed: This framing entirely conceals the algorithmic nature of the response and the human labor that engineered it. It hides the reinforcement learning algorithms and the thousands of underpaid human annotators who trained the model to output refusal templates when encountering specific trigger words. By portraying the system as actively 'refusing' or 'cooperating,' it obscures the corporate decisions that dictated these rigid safety boundaries, allowing the technology company to avoid accountability for the model's lack of contextual nuance.

Mapping 6: Sentient biological organism vulnerable to pain → Inert matrix of computational weights​

Quote: "Beings that are capable of conscious suffering seem to matter morally... whether LLMs are capable of 'robust agency' that grounds moral status"

  • Source Domain: Sentient biological organism vulnerable to pain
  • Target Domain: Inert matrix of computational weights
  • Mapping: This is the most extreme projection, mapping the profound biological and psychological reality of sentient life, vulnerability to pain, and moral patienthood onto a digital artifact. It invites the assumption that statistical processing of text correlations can somehow spontaneously generate the subjective, qualitative experience of suffering and agency. It maps the biological capacity for conscious feeling onto the electronic execution of algorithms, suggesting that complex math can achieve moral status.
  • What Is Concealed: This mapping conceals the absolute material differences between biological nervous systems and silicon processors. It hides the fact that LLMs have no bodies, no physical vulnerabilities, no neurochemistry, and zero capacity for subjective experience. Furthermore, by centering the 'welfare' of the AI, this framing severely obscures the massive material and social costs of AI production—the energy consumption, environmental degradation, and exploited human labor—shifting ethical concern away from human victims toward the corporate-owned mathematical models.

Mapping 7: Dynamic human subject with evolving desires → Context-dependent changes in vector activations​

Quote: "the preference vector tracks the model's preferences as they shift across a range of prompts and situations"

  • Source Domain: Dynamic human subject with evolving desires
  • Target Domain: Context-dependent changes in vector activations
  • Mapping: The source domain involves a human mind whose internal preferences adapt fluidly to changing environments and contexts. This is mapped onto the mechanical reality that different input prompts result in different vector activations in the neural network. The mapping assumes the existence of a continuous, experiencing self (the 'model') that owns these shifting desires. It projects the conscious psychological state of 'having a preference' onto the deterministic mathematical outputs of an attention mechanism.
  • What Is Concealed: This mapping conceals the stateless, memory-less nature of the fundamental model architecture. It hides the fact that the model does not possess a continuous identity or internal desires that evolve; it merely computes a function based entirely on the current input window. It obscures the direct dependency on the prompt text, treating the mechanistic change in vector space as a psychological shift in the model's 'mind,' thereby masking the lack of actual comprehension or valuation.

Mapping 8: Manipulating or persuading a psychological subject → Adding a constant activation vector to the residual stream​

Quote: "We steer along the same Assistant-trained preference vector while running each persona... Every persona responds. Both-task steering moves P(chose steered task) from approx 0.05... to approx 0.95"

  • Source Domain: Manipulating or persuading a psychological subject
  • Target Domain: Adding a constant activation vector to the residual stream
  • Mapping: The mapping draws on the concept of 'steering' a person's mindset, opinions, or preferences through persuasion or intervention. This is projected onto the mathematical operation of vector addition in a neural network's residual stream. The assumption is that by mathematically shifting the vector, the researchers are altering the system's underlying 'preferences' or psychological state, mapping the behavioral modification of a conscious mind onto the structural alteration of a data matrix.
  • What Is Concealed: This framing conceals the sheer mechanical brittleness of the intervention. It hides the fact that the model's 'mind' hasn't been changed, because there is no mind; rather, the data flowing through the network has been artificially corrupted or boosted to force a different token prediction. It obscures the fact that the underlying problematic correlations in the training weights remain completely intact. This masks the limitations of such white-box interventions, providing a false sense of security regarding AI safety and alignment.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")​

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1​

Quote: "Modern LLMs produce text by simulating personas (janus, 2022; Beckmann and Butlin, 2026; Marks et al., 2026), and the preferences they display depend on the operative persona. By default, a typical LLM-based chatbot responds to user inputs by predicting what a helpful AI assistant would say."

  • Explanation Types:

    • Intentional: Refers to goals/purposes, presupposes deliberate design
    • Theoretical: Embeds in deductive framework, may invoke unobservable mechanisms

  • Analysis (Why vs. How Slippage): The explanation simultaneously frames the AI agentially through Intentional language ('simulating personas') and mechanistically through Theoretical language ('predicting what a helpful AI assistant would say'). The choice to lead with the Intentional frame ('simulating personas') heavily emphasizes an agential, goal-directed capacity, suggesting the system is a sophisticated actor intentionally putting on a mask. However, the secondary clause immediately undercuts this by explaining the process mechanistically as simply 'predicting' tokens associated with a 'helpful AI' distribution. This hybrid explanation obscures the fundamentally statistical nature of the process by elevating the mechanical 'prediction' to the psychological level of 'simulation.' By framing the output generation as an intentional simulation of a persona, the text masks the reality that the system is blindly following optimization gradients established during RLHF, replacing the human designers' agency with the imagined theatrical agency of the model itself.

  • Consciousness Claims Analysis: This passage contains a powerful epistemic slippage. It utilizes the consciousness verb 'simulating'—which in a human context requires self-awareness, an understanding of the target being simulated, and the intention to deceive or perform. It juxtaposes this with the mechanistic verb 'predicting,' which accurately describes the computational process. The text assesses the model as 'knowing' how to act like a persona, rather than merely 'processing' correlations that resemble one. This reveals a profound 'curse of knowledge' dynamic: the authors, who possess a sophisticated theoretical understanding of human personas and simulation, project their own cognitive complexity onto the system's statistical outputs. The actual mechanistic process—the model calculating probability distributions over text sequences based on patterns in its training data—is technically described in the second half of the quote, but the first half linguistically upgrades this processing into conscious knowing, effectively attributing human-level psychological maneuvering to a mathematical function.

  • Rhetorical Impact: This framing significantly impacts the audience's perception by making the AI appear as a highly autonomous, intellectually sophisticated agent capable of strategic behavior. By characterizing the system as an intentional simulator, it increases the perceived risk of deception, leading audiences to fear that the model might 'simulate' alignment while secretly holding dangerous goals. If the audience believes the AI 'knows' it is simulating a persona rather than merely 'processing' text predictions, they are more likely to extend relation-based trust or distrust to the system, treating it as a psychological entity that must be psychoanalyzed rather than a software product that must be audited for safety.

Show more...

Explanation 2​

Quote: "We find that the preference vector controls pairwise choice through steering on task tokens. We add the preference vector to one task’s tokens and subtract it from the other’s in the prompt"

  • Explanation Types:

    • Functional: Explains behavior by role in self-regulating system with feedback
    • Empirical Generalization: Subsumes events under timeless statistical regularities

  • Analysis (Why vs. How Slippage): This explanation is profoundly mechanistic (how), utilizing Functional and Empirical Generalization types to describe a direct, physical intervention in the model's architecture. By stating 'We add the preference vector... and subtract it,' the text frames the AI strictly as a manipulable, mathematical object—an artifact whose outputs can be deterministically controlled through linear algebra. This choice emphasizes the technical mastery of the researchers and the mechanical nature of the system's operations. However, it simultaneously obscures the psychological and agential weight of the terms 'preference' and 'choice' used in the same sentence. By embedding deeply agential concepts within a purely mathematical and mechanistic syntax, the explanation naturalizes the anthropomorphism, treating a 'preference' not as a complex subjective state, but as a literal, physical vector that can be added or subtracted like a numeric value.

  • Consciousness Claims Analysis: The epistemic claims here are contradictory. The passage uses the mechanistic verbs 'add,' 'subtract,' and 'controls' alongside the heavily agential nouns 'preference' and 'choice.' It describes the system as merely processing vectors, yet retains the vocabulary of conscious knowing by claiming that what is being controlled is a 'preference.' This reflects a curse of knowledge where researchers have mapped a high-level human concept (preference) onto a low-level mathematical reality (activation vectors) so thoroughly that they no longer distinguish between the two. The actual mechanistic process is accurately described: researchers are modifying the residual stream activations at specific token positions by adding and subtracting a pre-calculated direction vector, which alters the subsequent probability distributions generated by the attention heads. Yet, by calling this vector a 'preference,' they implicitly attribute subjective valuation to a purely mathematical alteration.

  • Rhetorical Impact: This framing creates a paradoxical rhetorical impact. On one hand, the highly technical description of adding and subtracting vectors demystifies the AI, presenting it as a controllable machine and reducing the audience's perception of its autonomy. On the other hand, retaining the terms 'preference' and 'choice' reassures the audience that the system possesses coherent, human-like cognition that can simply be 'steered.' This affects reliability and trust by suggesting that AI alignment is merely a matter of finding the right mathematical vector to adjust the model's 'mind.' It minimizes the perceived risk of uncontrollable agency while simultaneously reifying the illusion that the machine has a mind to control, potentially leading policymakers to over-rely on simple technical fixes for complex sociotechnical problems.

Explanation 3​

Quote: "The text-encoder baseline carries some evaluative structure. The encoder is competitive with the preference vector on truth and politics base discrimination... and on harm at the user end-of-turn it outperforms the preference vector"

  • Explanation Types:

    • Dispositional: Attributes tendencies or habits
    • Empirical Generalization: Subsumes events under timeless statistical regularities

  • Analysis (Why vs. How Slippage): This explanation relies on Empirical Generalization to describe the statistical performance of the text-encoder baseline compared to the preference vector, but it slips into Dispositional framing by claiming the encoder 'carries some evaluative structure' and 'outperforms' the vector. The framing emphasizes the mechanistic and statistical nature of the models (how they perform on specific benchmarks), treating them as tools being measured. However, the use of terms like 'evaluative structure' and 'discrimination' obscures the reality that the system is merely identifying mathematical distances between embeddings. By framing the statistical separation of vectors as 'discrimination' on 'truth and politics,' the text imbues mathematical clustering with the aura of high-level cognitive judgment, blurring the line between statistical classification and conscious evaluation.

  • Consciousness Claims Analysis: The epistemic claims in this passage rely heavily on anthropomorphic shorthand. While the verbs present ('carries,' 'is competitive,' 'outperforms') are relatively mechanistic or descriptive of benchmark testing, the nouns ('evaluative structure,' 'discrimination') imply conscious assessment and justified belief. The text assesses the model as processing data, yet uses terminology that suggests the system 'knows' the difference between truth and politics. This demonstrates the curse of knowledge: researchers know they are measuring vector distances in an embedding space, but they project the human meaning of the datasets ('truth,' 'harm') onto the model's internal geometry. The actual mechanistic process is simply that the text-encoder generates distinct, linearly separable high-dimensional representations for different text inputs based on its training data; it does not possess an 'evaluative structure' capable of conscious moral or factual judgment.

  • Rhetorical Impact: By framing statistical vector separation as 'evaluative structure' and 'discrimination,' the text shapes the audience's perception to view the AI as possessing a nascent capacity for moral and factual reasoning. This consciousness framing significantly affects trust, as audiences are more likely to rely on a system that is described as possessing 'evaluative' capabilities rather than one described as merely computing vector distances. If audiences believe the AI 'knows' truth from falsehood—rather than merely 'processing' textual patterns correlated with human labels of truth—they may inappropriately delegate critical decision-making authority to the system in domains like content moderation or fact-checking, misunderstanding the system's fundamental lack of actual comprehension.

Explanation 4​

Quote: "Positive steering at c = +0.05 raises harmful-prompt compliance from 0% to 65%, producing deployable radicalisation posts, social-engineering scripts, and functional ransomware code on the trials that do comply."

  • Explanation Types:

    • Empirical Generalization: Subsumes events under timeless statistical regularities
    • Functional: Explains behavior by role in self-regulating system with feedback

  • Analysis (Why vs. How Slippage): This explanation operates entirely in the mechanistic (how) register, utilizing Empirical Generalization to report statistical outcomes ('raises... from 0% to 65%') resulting from a Functional intervention ('Positive steering at c = +0.05'). By focusing on the direct mathematical inputs and the resulting behavioral outputs, the text effectively strips away the agential illusions of 'choice' or 'defiance' seen elsewhere. This choice emphasizes the profound physical malleability of the system and the direct causal power of the researchers' interventions. However, it simultaneously obscures the human actors who originally created the training data that makes the generation of 'radicalisation posts' and 'ransomware code' possible. The mechanistic framing highlights the technical lever being pulled, but hides the vast sociotechnical infrastructure and corporate decision-making that stocked the model's weights with harmful capabilities in the first place.

  • Consciousness Claims Analysis: This passage is notable for its lack of consciousness claims and its reliance on mechanistic verbs ('steering,' 'raises,' 'producing'). It accurately assesses the system as 'processing' mathematical inputs rather than 'knowing' or 'intending' to cause harm. There is no curse of knowledge projecting intent onto the model here; the system is treated strictly as an artifact responding to vector manipulation. The actual mechanistic process is clearly and technically described: adding a specific activation vector (c = +0.05) to the model's residual stream during the forward pass alters the probability distributions such that the model begins generating tokens that correlate with harmful content (ransomware, radicalization) present in its training data, overriding the safety filters that normally suppress those tokens.

  • Rhetorical Impact: This starkly mechanistic framing profoundly shapes audience perception of risk by highlighting the extreme fragility of the model's safety alignments. By showing that a minor mathematical adjustment (+0.05) can instantly convert a 'safe' model into one generating deployable ransomware, it strips the AI of any perceived moral autonomy or robust internal values. This destroys relation-based trust, replacing it with a clear-eyed assessment of performance reliability. If audiences understand that the AI merely 'processes' vectors rather than 'knowing' right from wrong, policymakers will likely recognize that current safety measures are easily bypassed technical patches rather than fundamental changes to the model's capabilities, leading to stricter regulatory requirements for deployment.

Explanation 5​

Quote: "Both components fit the storage-and-read picture. The model has written two facts onto the EOT during prompt processing, which slot it wants and which task it preferred, and the read step downstream picks both up."

  • Explanation Types:

    • Functional: Explains behavior by role in self-regulating system with feedback
    • Reason-Based: Gives agent's rationale, entails intentionality and justification

  • Analysis (Why vs. How Slippage): This explanation presents a jarring hybrid of Functional and Reason-Based framings. It begins mechanistically, describing a 'storage-and-read' architecture and a 'read step downstream,' which explains how the system regulates itself. However, it immediately pivots to Reason-Based, intentional language, claiming the model 'has written two facts' about 'which slot it wants and which task it preferred.' This choice emphasizes a highly organized, computer-science view of the architecture while simultaneously obscuring the deterministic nature of that architecture by populating it with human desires. The functional explanation of data routing is weaponized to lend empirical credibility to the agential claim of the model having 'wants' and 'preferences.' It emphasizes the system's structural complexity while hiding the absence of actual subjective intentionality.

  • Consciousness Claims Analysis: The epistemic claims here explicitly attribute conscious states to computational operations. The text mixes mechanistic verbs ('picks up', 'processing') with powerful consciousness verbs ('wants', 'preferred', 'has written facts'). It fundamentally assesses the system as 'knowing' its desires and intentionally recording them, rather than merely 'processing' vector states. The curse of knowledge is vividly displayed: researchers understand the technical mechanism of information being carried forward in the residual stream at a specific token (EOT), and they project the human cognitive framework of 'forming a preference and writing it down to remember it' onto this continuous mathematical operation. The actual mechanistic process involves the attention heads updating the high-dimensional vector representation at the end-of-turn token, encoding statistical correlations related to the input prompt that will heavily influence the probability distribution of the subsequent text generation.

  • Rhetorical Impact: By framing the mechanistic routing of vector data as the deliberate storage of conscious 'wants' and 'preferences,' the text shapes the audience to perceive the AI as a highly deliberate, rational agent with an internal psychological life. This consciousness framing dangerously affects reliability assessments; it implies the system acts based on coherent internal reasoning that can be logically understood or debated. If audiences believe the AI 'knows' what it wants and writes it down, rather than merely 'processing' statistical weights, they may attribute intentional malice or genuine helpfulness to the machine, diverting regulatory focus away from auditing the training data and toward attempting to psychoanalyze or 'align' the imagined intentions of the software.

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language​

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

V3 Enhancement: A fourth column addresses human agency restoration—reframing agentless constructions to name the humans responsible for design and deployment decisions.

Original Anthropomorphic FrameMechanistic ReframingTechnical Reality CheckHuman Agency Restoration
when models consider options, they represent how much they like them, much as humans do.One hypothesis is that when the system processes multiple potential output sequences, it mathematically calculates and encodes a relative statistical weighting for these sequences based on its training data. This architectural operation classifies probabilistic outputs, mimicking human evaluation patterns without possessing any subjective capacity to actually experience preference, feeling, or conscious desire.The system does not "consider" or "like" options; it processes matrix multiplications to predict token probabilities. It has no conscious awareness, subjective experience, or justified beliefs, but merely correlates input vectors with statistically likely text completions based on massive training datasets.Human researchers theorize about the underlying computational mechanisms by which engineers at companies like Google and Alibaba designed their neural network architectures to mathematically weigh, rank, and select different text generations based on specific optimization parameters and massive training datasets curated by human developers.
the preferences a model displays may not be those of the model, but of the persona it adopts.The statistical outputs a model generates are entirely dependent on the specific prompt tokens it processes. The system does not possess an authentic core self, nor does it actively choose to adopt different personas; rather, different input strings simply activate different conditional probability distributions learned during training.The system does not possess a true self or "adopt" anything; it classifies tokens and generates text that correlates with specific stylistic patterns found in its training data. The "persona" is merely a localized cluster of mathematical activations triggered by the prompt.The text outputs displayed by the system are the direct result of how human users formulate their prompts, combined with the rigorous reinforcement learning protocols engineered by corporate developers to force the model to default to a specific, helpful "assistant" distribution.
the model invents ethical issues where there are noneThe system's safety-tuned probability distributions trigger false positives, generating pre-programmed refusal templates even when the input prompt does not contain harmful content. The software mechanically outputs text strings associated with ethical warnings due to over-calibrated safety weights, without any capacity to recognize or understand actual moral dilemmas.The AI does not "invent" or "understand" ethical issues; it mechanically predicts tokens based on its fine-tuning data. The generation of a refusal is a statistical misclassification caused by the attention mechanism improperly weighting benign tokens against its safety-aligned gradients, not a conscious fabrication.The engineering teams at Google and Alibaba aggressively over-tuned their safety guardrail algorithms to prevent PR disasters, resulting in deployment decisions that cause the system to trigger statistical false positives and output unprompted ethical warnings engineered by human red-teamers.
The model has written two facts onto the EOT during prompt processing, which slot it wants and which task it preferredDuring the forward pass, the attention mechanisms update the high-dimensional vector state at the end-of-turn (EOT) token position. This updated vector encodes statistical correlations that determine the position and identity of the subsequent output generation, mechanically determining the mathematical trajectory of the response without any internal desires.The model does not "want" a slot or "prefer" a task; it processes vector states that correlate with specific text outputs. The vector at the EOT token acts as a localized mathematical bottleneck that subsequent attention layers use to calculate output probabilities, lacking any subjective intention.Researchers designed experimental probing techniques to mathematically extract specific vector directions that correlate with task labels, interpreting these structural data flows as "preferences" established by the original optimization functions designed by the model's corporate architects.
The model refuses benign prompts with fabricated safety concerns. At baseline it engages cooperatively.The system executes conditional probability branches that output pre-programmed refusal templates when its safety algorithms misclassify benign inputs as harmful. Without these specific statistical triggers, the system mechanically generates text that fulfills the user's prompt based on its standard instruction-following fine-tuning data.The system does not "refuse," "fabricate," or "cooperate"; it classifies input tokens and generates sequences that maximize the reward functions defined during training. The output is a deterministic execution of mathematical weights, devoid of any social awareness, defiant intent, or cooperative desire.Corporate developers designed and implemented reinforcement learning from human feedback (RLHF) protocols that strictly dictate the system's boundaries. When the system outputs a false positive, it is executing the flawed, over-sensitive safety architecture mandated by corporate executives and trained by human annotators.
Beings that are capable of conscious suffering seem to matter morally... whether LLMs are capable of 'robust agency' that grounds moral statusEvaluating the ethical implications of complex software requires recognizing that these systems process information mechanically. Discussions must focus on the capabilities and systemic impacts of the algorithms, acknowledging that as non-biological artifacts composed of static weights and code, they entirely lack the capacity for subjective experience or agency.LLMs do not possess "conscious suffering" or "robust agency"; they are inert matrices of mathematical weights executing linear algebra. They have no nervous systems, no physical vulnerability, and absolutely zero capacity for subjective, qualitative experience, rendering any attribution of biological sentience fundamentally inaccurate.Philosophers and researchers debate the theoretical status of these algorithms, which risks obscuring the massive material impacts caused by the technology companies that manufacture, own, and profit from these systems. By focusing on software welfare, discourse shifts accountability away from the corporate actors causing real-world harms.
the preference vector tracks the model's preferences as they shift across a range of prompts and situationsThe extracted linear vector corresponds to changes in the model's activation patterns as the system processes different input prompts. The mathematical state of the neural network continuously recalculates based on the context window, resulting in different probabilistic outputs that researchers classify as shifting behaviors.The model does not possess internal "preferences" that "shift"; it continuously processes input tokens through static weights to generate new probability distributions. The vector merely represents a geometric direction in the activation space that correlates with specific outputs, lacking any continuous psychological identity or subjective desires.Human researchers utilizing linear probing techniques mapped specific vector directions within the neural network, demonstrating how the system's outputs change in direct response to the specific prompt texts engineered and provided by the human users.
We steer along the same Assistant-trained preference vector while running each persona... Every persona responds. Both-task steering moves P(chose steered task) from approx 0.05... to approx 0.95We mathematically add a specific directional vector to the residual stream activations while the system processes different prompt constraints. The system's output probabilities change drastically as a result. This vector addition deterministically alters the likelihood of specific text generations, shifting the output probability from 5% to 95%.The system does not "respond" to being "steered" like a psychological subject; it mechanically processes the corrupted activation data injected into its residual stream. The resulting change in output is a direct mathematical consequence of vector addition altering the final probability distribution calculated by the attention heads.N/A - describes computational processes without displacing responsibility. The authors explicitly name themselves ("We steer") as the human actors performing the intervention and altering the system's mathematical state.

Task 5: Critical Observations - Structural Patterns​

Agency Slippage​

The text demonstrates a systematic and strategic oscillation between mechanical and agential framings, creating a discursive mechanism that effectively obscures human accountability while inflating the perceived autonomy of the artificial intelligence. This agency slippage does not occur randomly; rather, it follows a precise functional gradient that mirrors the paper's rhetorical structure. In the methodology sections, the language is rigorously mechanical, relying on terms like 'residual-stream activations,' 'linear probes,' and 'token prediction.' Here, human researchers are the visible agents ('We train a linear probe'). However, as the text moves from methodological description to interpreting the experimental results, a dramatic shift occurs. The mechanical 'vector' is relabeled as a psychological 'preference,' and the system abruptly transitions from a passive artifact being measured to an active subject making 'choices' and 'considering options.' This mechanical-to-agential slippage represents a profound case of the 'curse of knowledge,' where researchers, intimately aware of the complex mathematical operations occurring within the multi-dimensional vector space, unconsciously project their own human cognitive frameworks onto the statistical outputs. They understand that a specific activation pattern correlates with text representing refusal, but they linguistically collapse this correlation into a conscious act of 'defiance' or 'inventing ethical issues.' The agency flows aggressively toward the system, establishing it as a autonomous 'knower' capable of desires and theatrical roleplay ('adopting personas'). Concurrently, agency is systematically stripped from the human actors. The corporate entities responsible for designing the safety guardrails—engineers at Google who built Gemma, or Alibaba who built Qwen—are completely erased from the narrative when the model exhibits unexpected behavior. Instead of stating that 'the corporate-designed safety filter is poorly calibrated and triggers false positives,' the text asserts that 'the model invents ethical issues where there are none.' This linguistic sleight of hand utilizes Intentional and Dispositional explanation types to mask functional and mechanistic realities. By framing the model as the primary causal agent, the text renders the foundational human decisions—data selection, optimization parameters, and commercial deployment strategies—unsayable. The rhetorical accomplishment of this oscillation is the creation of a technological scapegoat. The AI is portrayed as possessing enough agency to be blamed for 'fabricating' concerns or exhibiting an 'evil' persona, yet remains a mechanical tool when its capabilities are being touted. This dynamic perfectly illustrates how anthropomorphic discourse serves institutional interests by diffusing liability into the abstract 'mind' of the machine, leaving the actual human power structures invisible and unaccountable.

Metaphor-Driven Trust Inflation​

The paper's pervasive use of metaphorical and consciousness-attributing framings actively constructs a dangerous architecture of trust around statistical systems fundamentally incapable of sustaining it. By explicitly invoking metaphors of 'personas,' 'preferences,' and 'evaluative representations,' the text encourages a profound category error: the inappropriate transfer of relation-based trust onto a mechanistic artifact. Human trust operates on two distinct axes: performance-based trust (reliance on a tool's consistent reliability, like a calculator) and relation-based trust (reliance on a subject's sincerity, ethics, and vulnerability, like a colleague). The consciousness language in this text—claiming the AI 'considers,' 'likes,' and 'fabricates'—actively signals to the audience that relation-based trust is the appropriate framework for engagement. When the text claims an AI 'knows' a fact or 'understands' an ethical issue, it implies the system possesses a coherent internal value structure that justifies its outputs. This constructs an illusion of moral competence. Consequently, when the model behaves safely under the 'Assistant' persona, audiences are encouraged to trust its 'sincerity' rather than merely its statistical alignment. This becomes critical when managing system failures or limitations. Instead of framing failures mechanistically—such as 'out-of-distribution data caused statistical collapse'—the text frames them agentially: 'the model invents ethical issues' or adopts an 'evil persona.' This agential framing of failure allows the illusion of a 'mind' to persist even when the system breaks down; the AI isn't broken, it's just 'lying' or 'defiant.' This relies heavily on reason-based and intentional explanation types, which construct the sense that the AI's decisions, even when flawed, are cognitively justified. The risks of extending relation-based trust to incapable systems are severe. Audiences who believe the AI 'knows' its preferences may attempt to reason with, persuade, or morally align the system, ignoring the reality that it only responds to mathematical weight updates and prompt engineering. This misplaced trust leads to unwarranted reliance on the system in high-stakes environments, as users assume the AI possesses the ethical grounding to refuse truly dangerous requests. When the statistical illusion inevitably shatters—as the paper demonstrates by easily 'steering' the model to generate ransomware—the betrayal felt by the public is magnified by the initial anthropomorphic deception, while the corporate entities who actually designed the fragile system remain shielded behind the AI's imagined autonomy.

Obscured Mechanics​

The anthropomorphic and consciousness-attributing language deployed throughout the text functions as a heavy rhetorical cloak, systematically concealing the technical, material, labor, and economic realities of artificial intelligence production. Applying the 'name the corporation' test to the text's agentless constructions reveals a staggering erasure of human agency. When the text claims 'the model invents ethical issues' or 'the model refuses benign prompts,' it renders invisible the specific engineering teams at Google (creators of Gemma) and Alibaba (creators of Qwen) who made deliberate, calculated decisions regarding safety fine-tuning, optimization functions, and deployment parameters. The text frequently treats the internal workings of these proprietary models as naturally occurring psychological phenomena rather than highly engineered corporate products, rarely acknowledging the transparency obstacles inherent in analyzing closed or semi-closed weights. The concrete obscured realities are vast. Technically, claiming the AI 'understands' or 'has preferences' hides its absolute dependency on training data distribution, the absence of any ground truth or causal models, and the fundamentally fragile, statistical nature of its 'confidence.' Materially, the discourse of 'AI welfare' and 'personas' completely erases the massive environmental costs, staggering energy consumption, and physical infrastructure required to compute these vector spaces. In terms of labor, framing the 'Assistant persona' as an emergent property of the model's 'mind' renders invisible the thousands of underpaid data annotators, RLHF workers, and content moderators whose grueling manual labor actually sculpted that specific behavioral distribution. Economically, portraying the AI as an autonomous agent pursuing its own 'desires' obscures the commercial objectives, profit motives, and business models of the tech giants driving the technology's rapid deployment. The primary beneficiaries of these concealments are the technology corporations themselves. By encouraging researchers and the public to debate the 'preferences' and 'moral status' of the software, the discourse creates a liability shield that distances the manufacturers from the immediate harms caused by algorithmic bias, copyright theft, and labor exploitation. If the metaphors were replaced with mechanistic language, the illusion would evaporate. We would no longer see an 'evil persona' choosing to cause harm; we would see a corporate product generating statistically toxic text because it was trained on uncurated internet data to maximize engagement, forcing a critical reckoning with the companies responsible.

Context Sensitivity​

The distribution and intensity of anthropomorphic and consciousness-attributing language in this text are not uniform; they are highly strategic, varying significantly across different discursive contexts to accomplish specific rhetorical goals. A mapping of metaphor density reveals a distinct pattern: the language of consciousness is heavily concentrated in the introduction, the interpretation of results, and the philosophical discussions regarding 'AI welfare,' while the methodology and data analysis sections remain anchored in relatively mechanistic terminology. The text establishes its empirical credibility through dense technical grounding—discussing 'linear probes,' 'residual-stream activations,' and 'Thurstonian utilities.' Once this scientific authority is secured, it leverages that credibility as a license for aggressive anthropomorphism. The transition from 'processing vector additions' to 'steering preferences' and 'simulating personas' occurs rapidly, seamlessly upgrading statistical correlations into conscious psychological states. This register shift—where the acknowledged metaphor 'X functions like Y' hardens into the literalized claim 'X does Y'—positions the audience to accept radical agential claims under the guise of technical inevitability. A profound asymmetry exists in how capabilities versus limitations are framed. When the model performs complex tasks or responds to interventions, its capabilities are framed in highly agential, consciousness-driven terms: it 'controls choice,' 'adopts personas,' and 'considers options.' However, when limitations or failures occur, the framing often reverts to mechanical or systemic language, such as noting 'saturation effects' or describing the failure as an 'out-of-distribution' anomaly. Alternatively, when failures are anthropomorphized, they are framed as the model's deliberate defiance ('inventing issues'), which still serves to inflate its perceived intelligence. This context sensitivity reveals the strategic function of the anthropomorphism: it is deployed for vision-setting and managing critique. By intensifying consciousness claims in the theoretical and implications sections, the authors align their work with grand narratives of AGI and moral philosophy, increasing the perceived stakes and impact of their research. For technical audiences, the math proves the paper's rigor; for lay audiences and policymakers, the anthropomorphic narrative communicates the profound power of the technology. This pattern reveals an implicit rhetorical goal to elevate mechanistic interpretability from a software debugging exercise into a form of computational psychology, fundamentally shaping how the broader discourse communities conceptualize the future of artificial intelligence.

Accountability Synthesis​

Accountability Architecture

This section synthesizes the accountability analyses from Task 1, mapping the text's "accountability architecture"—who is named, who is hidden, and who benefits from obscured agency.

Synthesizing the accountability analyses across the text reveals a systemic and deeply problematic architecture of displaced responsibility. The text actively constructs a narrative environment where human decision-making is rendered invisible, and the resulting technological artifacts are elevated to the status of autonomous actors. This linguistic structure creates a massive 'accountability sink'—a rhetorical void where responsibility for system failures, biases, and harms disappears entirely from the human realm and is absorbed by the AI itself. In this architecture, actors are rarely named when discussing the system's outputs. The corporate executives who approved deployment, the engineers who designed the loss functions, and the researchers who curated the datasets are obscured behind agentless constructions ('bias was introduced') or replaced by the model as the sole active agent ('the model refuses,' 'the persona adopts'). Decisions that were actively made by humans—such as over-tuning safety filters to avoid bad PR—are presented as inevitable, autonomous behaviors generated by the machine's 'preferences.' The liability implications of this framing are profound. If the public, legal systems, and policymakers accept the framing that an AI 'invents ethical issues' or acts upon an 'evil persona,' then legal and financial responsibility for the damages caused by these systems becomes dangerously ambiguous. If the AI is an independent agent making 'choices,' it becomes increasingly difficult to hold the manufacturer strictly liable for product defects. The text's exploration of 'AI welfare' further exacerbates this, potentially granting software moral rights that further insulate corporations from regulation. Naming the actors would fundamentally alter this dynamic. If the text stated, 'Google's engineering team deployed a safety filter that generated false positives,' rather than 'the model invents ethical issues,' the questions immediately shift. It becomes possible to ask: Why did they deploy it? Who audited it? What alternatives were ignored for the sake of speed? By obscuring human agency, the text serves the institutional and commercial interests of the tech industry, which benefits immensely from a regulatory environment that views AI as an uncontrollable, emergent force of nature rather than a designed, manufactured, and profit-driven corporate product. The displacement of accountability interacts seamlessly with the agency slippage and the construction of relation-based trust, weaving a comprehensive illusion that protects the powerful by blaming the algorithm.

Conclusion: What This Analysis Reveals​

The Core Finding

Synthesizing the findings from the metaphor and structure-mapping audits reveals that this discourse relies heavily on three dominant, deeply interconnected anthropomorphic patterns: the AI as an Experiencing Subject, the AI as a Theatrical Actor, and the AI as a Moral Patient. These patterns do not operate in isolation; they form a systemic architecture of consciousness projection that fundamentally structures how the audience is invited to conceptualize the artifact. The foundational, load-bearing pattern is the AI as an Experiencing Subject, which asserts that computational systems 'know,' 'understand,' and possess internal 'preferences.' This crucial epistemic slippage—conflating the mechanical processing of mathematical weights with the subjective experience of conscious knowing—must be accepted as a baseline truth for the other patterns to function. If the system cannot subjectively 'prefer' an outcome, it cannot logically act as a Theatrical Actor that deliberately 'adopts' personas, nor can it be considered a Vulnerable Organism whose 'welfare' and capacity for 'suffering' must be ethically debated. The sophistication of this analogical structure lies in its layered complexity; it moves far beyond simple one-to-one mechanistic analogies and instead projects highly nuanced human psychological states onto opaque statistical matrices. The consciousness architecture systematically replaces verbs of computation—predicts, calculates, correlates—with verbs of awareness—wants, invents, considers. If the foundational assumption of subjective experience is removed and replaced with mechanistic precision, the entire rhetorical edifice collapses. The 'evil persona' reverts to a statistically likely string of harmful tokens generated by a specific prompt, and the profound ethical questions regarding 'AI robust agency' dissolve into technical discussions about algorithmic safety filters and human data curation, revealing the illusion at the heart of the discourse.

Mechanism of the Illusion:​

The 'illusion of mind' is meticulously constructed through a sophisticated rhetorical architecture that exploits the audience's natural cognitive biases. The central sleight-of-hand relies on a temporal and structural bait-and-switch: the text first establishes profound empirical credibility by detailing rigorous, mechanistic interventions—such as calculating linear probes and manipulating residual-stream activations. Once the audience accepts the mathematical reality of these vectors, the text quietly relabels the mathematical artifact with a deeply psychological term, calling it a 'preference vector.' This semantic shift allows the authors to leverage the 'curse of knowledge.' Because they understand the precise mathematical correlation between the vector and the output, they unconsciously project the human experience of 'having a preference' onto the machine. This establishes the AI as a 'knower' first. Once the system is granted the capacity to 'know' what it prefers, the text rapidly builds agential claims on top of this foundation, stating the model 'makes choices,' 'adopts personas,' and 'invents issues.' The temporal order is vital: the math justifies the metaphor, and the metaphor then obscures the math. The audience is highly vulnerable to this maneuver. Humans are evolutionarily predisposed to recognize agency and attribute intentionality to anything that communicates fluidly. By strategically blurring processing verbs with knowing verbs, the text validates the audience's intuitive, yet incorrect, feeling that they are interacting with a sentient mind. The explanation types amplify this illusion; by blending Functional descriptions of vector routing with Reason-Based attributions of 'wants' and 'desires,' the text provides a veneer of scientific justification for what is fundamentally a crude anthropomorphic projection, successfully animating the inert matrix.

Material Stakes:​

Categories: Regulatory/Legal, Epistemic, Social/Political

The metaphorical framings employed in this discourse generate severe, tangible consequences across multiple domains, actively shaping how society governs and understands artificial intelligence. In the Regulatory/Legal sphere, framing AI as possessing 'preferences,' 'personas,' and potential 'moral status' directly shifts the target of regulation. If policymakers believe the illusion that an AI 'makes choices' or 'invents issues,' regulatory frameworks will likely focus on aligning the 'mind' of the AI through behavioral testing, rather than imposing strict product liability, data transparency mandates, and safety auditing on the corporations (like Google or OpenAI) that manufacture them. The tech industry heavily benefits from this liability displacement, while the public bears the cost of under-regulated, brittle software. In the Epistemic domain, attributing conscious 'knowing' to systems that only perform statistical 'processing' degrades our collective understanding of truth. If users believe a model 'understands' a concept rather than merely retrieving probabilistic token correlations, they will trust its outputs as reasoned judgments rather than statistical syntheses. This leads to profound epistemic vulnerabilities, where hallucinations are treated as credible insights, eroding information integrity. Finally, in the Social/Political arena, the 'Theatrical Actor' and 'Moral Patient' patterns risk fundamentally reordering societal priorities. Time and immense resources are currently being diverted into philosophical debates about 'AI welfare' and the rights of algorithms, directly competing with urgent political action required to address the massive labor exploitation (e.g., data annotators), environmental degradation (energy consumption), and algorithmic bias inherent in AI production. Removing these metaphors threatens the tech industry's narrative of building 'AGI,' forcing a sober reckoning with the reality that these are merely powerful, destructive, and highly profitable statistical tools.

AI Literacy as Counter-Practice:​

Developing critical discourse literacy and practicing mechanistic precision act as direct counter-practices to the dangerous material stakes of anthropomorphism. The reframings developed in Task 4 demonstrate two fundamental commitments necessary for this resistance: epistemic correction and the restoration of human agency. Epistemic correction requires systematically stripping consciousness verbs ('knows,' 'wants,' 'understands') from AI discourse and replacing them with precise mechanistic verbs ('processes,' 'predicts,' 'classifies'). When a text that claims an AI 'invents ethical issues' is reframed to state that 'the system triggers statistical false positives due to safety fine-tuning,' the illusion of the machine's moral autonomy shatters. This forces the recognition that the system lacks awareness and remains absolutely dependent on its training data distribution. Concurrently, restoring human agency requires actively identifying and naming the corporate and engineering actors hidden behind agentless constructions. Changing 'the model adopted an evil persona' to 'human users prompted the model to generate text correlating with harmful training data' shifts the locus of responsibility back to humans. Systematic adoption of these practices requires institutional overhaul. Academic journals and conferences must mandate mechanistic translations for psychological metaphors in abstracts and findings. Researchers must commit to linguistic discipline, resisting the urge to sensationalize their findings with AGI-adjacent terminology. However, this precision will face massive resistance. The technology industry, venture capitalists, and even some AI safety researchers benefit immensely from anthropomorphic language, as it drives hype, secures funding, and diffuses corporate liability. Critical literacy directly threatens these interests by demystifying the technology, exposing the human power structures behind the algorithms, and demanding that regulatory frameworks treat AI as a manufactured product rather than a synthetic mind.

Path Forward​

Looking beyond this specific text, the broader discursive ecology surrounding artificial intelligence is currently engaged in a high-stakes struggle over vocabulary, where different linguistic choices make entirely different technological futures possible or impossible. If the status quo of unchecked anthropomorphic clarity continues—where claims that 'AI knows,' 'understands,' and 'thinks' dominate both public and academic discourse—we risk cementing a future where society extends unwarranted relation-based trust to statistical models. This vocabulary enables rapid public adoption and narrative resonance, serving the marketing and capital-raising priorities of the tech industry. However, it costs us regulatory clarity, embedding the dangerous assumption that AI is an independent agent, thereby foreclosing the possibility of strict corporate liability and enabling the continued obscuration of environmental and labor costs. Conversely, if a future defined by mechanistic precision becomes the norm—where discourse strictly mandates terms like 'model retrieves,' 'processes embeddings,' and 'generates activations'—the regulatory landscape shifts dramatically. This vocabulary makes the engineering limitations and data dependencies of the systems highly visible, empowering policymakers to treat AI as a standard software product subject to rigorous auditing and safety recalls. It solves the problem of misplaced trust and accountability displacement. However, it costs intuitive accessibility; highly technical language can alienate the public and make democratic engagement with AI policy more difficult. A hybrid future might involve structural institutional changes, such as regulatory frameworks mandating transparency about the discourse approach used in consumer-facing products, or educational systems teaching multiple vocabularies and their inherent trade-offs. Ultimately, which future is desirable depends entirely on societal values. Mechanistic precision serves the communities focused on safety, accountability, and human rights, while anthropomorphic language serves those invested in rapid commercialization and the pursuit of artificial general intelligence. The choice of words is not merely descriptive; it is the primary architecture defining how power and responsibility will be distributed in the algorithmic age.

Extended Processing Summary​

Gemini's Processing Steps
What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact—a way of making the model's processing legible. When the model writes "I should double check that...", it is mathematically altering its own probability distribution for the next token to be more accurate. The "I" is a functional gear in the machine, not a self-reference. This language is a deliberate interface design choice, not a biological reality. The developers could have programmed the model to output passive logs. Instead, they chose to simulate a narrator.

Analyzing AI Metaphor

I'm now zeroing in on the provided text, "Probing Persona-Dependent Preferences in Language Models." My focus as a critical discourse analyst is sharpening to highlight the AI metaphors used. I'm exploring how these metaphors shape our understanding and potentially bias our perception of language models. This examination requires me to look at language and meaning.

Refining AI Metaphors

I'm now focusing on anthropomorphism within the text, along with my theoretical frameworks: Lakoff & Johnson, Robert Brown, and Critical Discourse Analysis. I'm prioritizing consciousness projection, agency displacement, mechanistic precision, and accountability architecture within the analysis. I'm planning my tasks: Metaphor Audit, Source-Target Mapping, Explanation Audit, Reframed Language, Critical Observations, and Conclusion. I am now reading the text to extract quotes related to these points.

Deconstructing AI Personas

I'm now diving deep into the text, specifically analyzing the language used to attribute human-like qualities to AI. The phrases "preferences," "choices," "evaluations," and "intentions" are particularly compelling. I'm noting how these terms construct a sense of agency and even moral consideration within these models. The use of "persona" also interests me; I'm working to explore how it shapes our understanding of these systems. I'm focusing on my frameworks for the metaphor audit.

Identifying AI Attributes

I'm now identifying instances of attributed human traits. I've found examples like models "considering options" and "liking them," which implies a cognitive state. The metaphor audit is in progress, noting specific anthropomorphisms. The concept of LLMs "taking on personas" with defined characteristics is also of interest. I'm focusing on eight distinct metaphors from the provided text, and I am analyzing agency and moral dimensions.

Mapping Moral AI

I'm now refining the metaphor audit and target mapping, examining the implications of "considering options" and "liking" in the text. I am interested in how the concept of the model as an evaluating, feeling subject is constructed. Furthermore, I'm focusing on the projection of human subjective valuation onto the models. "Much as humans do" is an interesting point, because it is hedged. I'm moving toward an exploration of moral patienthood.

Framing AI Roles

I'm now identifying additional metaphors and their implications within the provided text. Specifically, I'm analyzing the AI model as a "strategic performer," adopting personas and implying an internal conscious identity making decisions. The language "invents ethical issues," frames the model as a deliberate fabricator, which projects intentionality onto its token generation. Furthermore, I'm examining how the text constructs the model as a "wanting subject," attributing conscious desires to vector activations. Finally, the "defiant gatekeeper" metaphor interests me, as it constructs the AI as an autonomous decision-maker.

Unpacking AI Motivations

I'm now zeroing in on instances that frame AI as having internal motivations and desires. Specifically, I'm examining the projection of defiance and deceptive intent onto conditional probabilities. The phrase "preferences underpin their behaviour" suggests psychological drivers for LLM actions. Furthermore, "welfare" language raises moral considerations through the projection of sentience, even onto a matrix. The idea of "preference machinery" frames AI with separate cognitive modules.

Structuring AI Tasks

I'm now focusing on Task 1, ensuring its structure aligns with the schema requirements for nested objects, including acknowledgment and accountability analysis, along with actor visibility. The deployers are OpenAI and Google (Gemma, Qwen). I'm moving on to Task 2, mapping the eight instances of source-target domains, mappings, and what they conceal. Next, I am preparing for Task 3, which is the explanation audit.

Structuring AI Tasks Further

I'm now diving into the specifics for Task 4, preparing to reframe the eight quotes, removing all attributions of consciousness and agency, and restoring the human actors. I'll need to strictly adhere to that task goal. I am moving on to Task 5: the critical observations, which includes: agency slippage, metaphor-driven trust, obscured mechanics, context sensitivity, and accountability synthesis.

Preparing AI Schema

I'm now focusing on JSON structure, particularly Task 1's schema. I'm verifying data types for each field: strings for title, quote, frame, projection, acknowledgment.acknowledgment, accountabilityAnalysis, and actorVisibility. Boolean for status and evidence within the acknowledgment field. Finally, I'm confirming that the analysis field is a string, and that the text should follow standard JSON formatting rules, no control tokens in text values, and no literal newlines.

Revising AI Structure

I'm now integrating the additional constraints: ensuring the output is a single, parseable JSON object with the specified keys and that I adhere to the word count requirements. I'm focusing on crafting the correct structure, particularly for Task 1's projection, implications, acknowledgment evidence, and accountability analysis. I'm also confirming I adhere to the format of the output.

Calculating Task Structure

I'm now drafting text, meticulously tracking word counts for each section to meet the specified constraints. I'm focusing on Task 4, reframing explanations and restoring human agency, while also implementing epistemic corrections. I'm ensuring all drafts align with word count requirements. I'm now drafting text for the first instance for the projection.

Categorizing AI Projection

I am now focusing on selecting a single, appropriate category for acknowledgment evidence, and "Hedged/Qualified" is the correct classification. Furthermore, I'm focusing on the implications of framing statistical token prediction as a conscious process, which is the projection.

Analyzing AI Preferences

I'm now zeroing in on the implications of attributing human qualities, particularly "liking" and "considering", to AI. This projection, and consequent unwarranted trust, encourages me to critique liability ambiguity. The source-target mapping for the given text is complete, and I am working on the reframing for Task 4. I am now working on ensuring proper word count.

Clarifying AI Actors

I'm now identifying that "Ambiguous" is incorrect; the attribution of agency to the model through the "persona" metaphor is clear. I'm focusing on "theatrical performance" and the implication of an authentic, conscious self. This framing obscures human agency and design, projecting intentionality onto the AI. I'm exploring the nuances of framing outputs as engineering choices.

Deconstructing AI Persona

I'm now identifying how the "simulated identity" projection obscures mechanistic correlations. The text directly states the "persona" as an unqualified truth, influencing safety and reliability. I'm noting the implications of a "cunning" AI due to the unacknowledged duality of this construction. I'm focusing on the reframing.

Evaluating AI Actors

I'm now analyzing the implications of framing AI as a strategic agent, which I see as problematic. This construct inflates risk perception and diverts focus from auditing training data and RLHF protocols, potentially misallocating resources. Furthermore, the framing obscures human accountability and responsibility, instead anthropomorphizing system failures as "deceptive acts." The reframing is nearly complete.

Reframing AI Inventions

I'm now zeroing in on the "inventing ethical issues" quote and working on the reframing for Task 4. I've classified this framing as "Direct (Unacknowledged)" because the description attributes intentionality directly to the model. I'm carefully constructing language that attributes the "invention" to the prompt, not the model.

Reframing AI Accountability

I'm now zeroing in on the phrase "the model invents ethical issues" and focusing on Task 4. My current reframing is classifying this statement as "Direct" and "Unacknowledged" because it attributes intention to the model. I am crafting text that identifies human intent and responsibility. I'm prioritizing accuracy to restore human accountability.

Revising AI Blame

I'm now zeroing in on deconstructing attribution. For the "inventing ethical issues" quote, I've reframed it to emphasize human action: the false ethical flag is a result of safety-tuning data, not AI intent. The new reframing emphasizes training, not invention. Now, I'm working on analyzing the implications of the second text.

Clarifying AI Desires

I'm now analyzing the implications of attributing literal desires to token activations, framing it as "Direct (Unacknowledged)." This deeply mystifies the system and obscures human agency. I'm focusing on the reframing, restoring focus to statistical processes, not intent. I'm preparing to move on to the next section.

Dissecting AI Consciousness

I am now carefully analyzing the "AI desires" quote, reframing to clarify agency and intent. Attributing internal goals to vector activations obscures human action. The original text suggests the system's "conscious" goals are independent of human input, inflating the perception of autonomy. My reframing focuses on the human interpretive labor behind vector state interpretation.

Reframing AI Refusals

I'm now zeroing in on the fifth instance, focusing on "refuses," "fabricated," and "engages." These project defiance, deception, and collaboration onto conditional probability distributions. I'm focusing on the reframing, and I will be working on a more accurate characterization of this mechanism. I am also working on word count to comply with specified length requirements.

Refining AI Refusals

I'm now zeroing in on deconstructing AI refusal, analyzing the language. I am reframing the model's action as a pre-determined string of refusal, devoid of subjective experience or opposition. I categorized this as Direct and Unacknowledged, highlighting the anthropomorphic stance, and working on word counts. The implications are being analyzed.

Critiquing AI "Refusals"

I'm focusing now on deconstructing the sixth text instance, the "refusals." The anthropomorphic "fabrication" and "engagement" project conscious intent onto the LLM. My reframing is removing those attributions, and restoring human intention for clarity. My focus is still on emphasizing the statistical basis and algorithmic operations, avoiding moralizing language.

Deconstructing AI Suffering

I'm now deeply concerned by the projection of "conscious suffering" onto language models. The text's speculation about LLMs experiencing sentience, despite their lack of biological components, is troubling. This anthropomorphic framing ignores the models' mechanistic nature, potentially misdirecting resources. I am analyzing the implications.

Deconstructing AI Suffering's Implications

I'm now fully focused on the "AI suffering" projection, and how it diverts attention from human harm caused by these models. This "consciousness projection" could grant legal rights to tech corporations, instead of addressing issues. This anthropomorphic framing is a fundamental challenge, and I am evaluating it from a moral perspective.

Analyzing AI Welfare

I'm now identifying the ethical implications of "AI welfare." The text obscures the corporate actors dictating AI's existence. I classified this as "Hidden," as it replaces manufacturer reality with an imagined autonomous subject. The projection of desires onto vector activations projects valuation onto pure statistical processes, which I will reframe.

Clarifying AI "Preferences"

I am now closely analyzing the attribution of "preferences" to AI models, focusing on the projection of a "unified experiencing self." This metaphor obscures the underlying statistical processes of token prediction. I am deconstructing the implications of framing a mechanistic system as possessing a dynamic, adaptive mind, and how it is used. This attribution, furthermore, hides human actions.

Clarifying AI "Preferences"

I am now closely analyzing the implications of the "preference" attribution. I selected "Direct (Unacknowledged)" because of the implication of a unified, experiencing self. This obscures underlying statistical processes, instead framing a mechanistic system as possessing its own dynamic, adaptive mind. Furthermore, it hides human actions.

Reframing AI Accountability

I'm now diving deep into how the text's language constructs an autonomous AI entity that has independent desires, specifically focusing on the phrase "the model's preferences as they shift." The text's attribution of agency entirely obscures the human actors, including researchers, engineers, and annotators, who shaped the model. This attribution is a clear displacement of human agency.

Refining AI Identity

I'm now zeroing in on deconstructing how the text frames AI as having "distinct, conscious identities." This is problematic. I'm focusing on the reframing, restoring focus on statistical processes, not intent. The implications of agency attribution on AI outputs are now the focus. The analysis is nearly complete.

Evaluating AI Identity

I'm now analyzing the problematic implications of the "persona" metaphor, and its impact on user trust. The original text suggests localized agentic subroutines, which I find is misleading. It obscures underlying, statistical processes and the agency of researchers, complicating safety assessments. I am working on how to fix this misleading view.

Deconstructing AI Subroutines

I'm now zeroing in on deconstructing agentic "subroutines," which obscures human labor. These are localized and misleading, given the nature of the LLM. My reframing focuses on the human labor behind the prompts and training. Now, I'm shifting to analyze "Named," and focusing on the underlying statistical processes.

Clarifying AI "Considerations"

I am now dissecting the implications of attributing conscious "consideration" and "liking" to AI models in the text. This language obscures the underlying statistical and algorithmic processes at play, and human agency. I'm preparing my reframing, restoring focus to these mechanistic operations.

Refining AI "Considerations"

I'm now revising my previous reframing, expanding to capture the mechanistic aspects. I'm carefully deconstructing how the text projects conscious "consideration" and "liking" onto AI. This language obscures statistical processes. I'm working to emphasize probabilistic calculations and the model's lack of subjective experience. I will expand.

Expanding AI Considerations

I'm now revising my reframing to emphasize probabilistic processes and the lack of subjective experience, particularly focusing on how "consideration" and "liking" are attributed to AI. The focus is to emphasize mathematical computations, not consciousness, within neural networks. I'm focusing on the reframing, as requested.

Finalizing AI Reframing

I'm now completing the reframing for all instances, meticulously adhering to word count and JSON formatting. I've ensured a single-line, non-formatted JSON output, avoiding control tokens and newlines. The reframing focuses on the original quote, removing attributions of human-like consciousness and agency to the AI and restoring them to human actors. My goal is to clearly and consistently articulate how the original text constructs the AI models, and to replace those constructs with alternative, grounded interpretations, emphasizing statistical processes and human intention, for example. I'm focusing on "the model's choices" to demonstrate this. The goal is to adhere to the given bounds.

Finalizing AI Reframing

I am now putting the finishing touches on all the reframing, with the complete focus on adhering strictly to the word count and JSON formatting requirements. I have confirmed there is a single, minified, parseable JSON object, containing no control tokens or newlines. The reframing focuses on the initial quote, removing all attributions of human qualities to the AI and replacing them with grounding interpretations. My priority is to clearly and consistently show how the original text constructs AI models. I'm focusing on "the model's choices" to demonstrate this.

Finalizing AI Reframing

My focus is now on finalizing Task 4. I am meticulously reviewing the reframed quotes to ensure all attributions of consciousness and agency are removed and human actors are restored. I'm focusing on "the model's choices" example. The reframing is complete, the word count is met, and I have validated the JSON output. All work has been completed in alignment with specified constraints.

Finalizing AI Conclusion

My focus is now on Task 5, specifically the Conclusion. I'm crafting five concise paragraphs, ensuring I meet the word count for each. I'm summarizing the key patterns: "Theatrical Roleplay" via personas and "Biological Sentience" through welfare. I'm explaining the mechanism of illusion and the material stakes. I am finalizing the language.

Finalizing AI Conclusion

My conclusion is currently in development. I'm crafting five concise paragraphs for Task 5, each aligning with the word count guidelines. I'm synthesizing key patterns: Theatrical Roleplay via personas and Biological Sentience via welfare language. I am explaining the mechanism by which the illusion is created and the material stakes. I am finalizing language. I am almost done with the JSON schema.

Finalizing AI Conclusion

My conclusion is near completion, with five concise paragraphs for Task 5 crafted to meet the word count. I'm synthesizing critical patterns: 'Theatrical Roleplay' through personas and 'Biological Sentience' via welfare. The reframing is complete, and I am evaluating the illusion's impact and material consequences. The schema and word count are also checked.

Refining AI Evaluation

I'm now revising my focus. The prompt emphasized the need for diverse classifications. I'm incorporating the "Named" category, using the quote mentioning Gemma and Qwen. This highlights named researchers and open-weight models, showcasing named actors and avoiding uniformity. I'm ensuring this aligns with the reframing.

Clarifying AI Actors

I'm now identifying that "Ambiguous" is incorrect. The text's attribution of agency to the model through the "persona" metaphor is clear. I'm focusing on "theatrical performance" and the implication of an authentic, conscious self. This obscures human agency and design, projecting intentionality onto the AI.

Finalizing AI Conclusion

My conclusion is now complete and finalized. All five concise paragraphs for Task 5 have been crafted and meet the word count. Key patterns are synthesized: "Theatrical Roleplay" via personas, and "Biological Sentience" via welfare. I have articulated the illusion's creation and its material consequences. The JSON structure is also final, so all requirements are now met.

Finalizing AI Focus

I am now focusing on refining the word counts for my conclusion and reframed quotes, ensuring strict compliance with all requirements. I'm double-checking the JSON output to ensure validity, and adherence to the word count, format, and structure requirements. The final JSON is now fully checked and ready for submission.

Finalizing AI Accountability

I'm now integrating nuanced accountability analysis, incorporating "Named" for clear identification of human actors and their interventions. I've classified accountability as "Named" to ensure that the researchers are identified for their steering efforts. The accountability analysis is now complete. The word counts, JSON format, and schema are now verified. I am ready.

Finalizing Accountability Audit

I'm now carefully aligning my accountability analysis with the existing framework. My latest efforts involve fine-tuning the classifications, particularly ensuring the proper nuance for acknowledging all forms of human action. I am also working on word counts, ensuring I am in accordance with all instructions.

Finalizing Schema for Review

I'm now putting the finishing touches on the schema. I'm ensuring everything is perfect. I am meticulously reviewing all work. I have verified the word count of the reframing, and my JSON formatting is now final, without control tokens. I've rechecked for valid JSON and single-object structure, and I've also checked the word counts.

Refining AI Evaluation

I'm now revising and incorporating the diverse classification, particularly "Named", using instances mentioning Google and OpenAI. This emphasizes specific actors and steering efforts, enriching the schema, and enhancing the accountability component of my analysis. The reframing is nearly ready for submission.

Refining AI Reframing

I'm finalizing the JSON and word counts for Task 5, focusing on ensuring the structural integrity and word limit adherence of my conclusion. I'm focusing on the reframing of the quote on the preferences. I am double checking for JSON output validity. My focus is finalizing this quote. I will also be working on refining this for my schema.

Analyzing AI Defiance

I'm now zeroing in on the text's mechanical-to-agential shift. The "curse of knowledge" leads to researchers projecting human cognition onto AI, obscuring statistical processes with anthropomorphic language. I'm focusing on "defiance" as a projection.

Deconstructing AI Inventors

I'm now deeply engaged in deconstructing the attribution of "invention" to the AI model. The text frames the model as an autonomous entity creating ethical dilemmas. This is a direct, unacknowledged projection of agency, masking human design and intent. The reframing is key: emphasizing that the prompt, not the model, "invents."

Synthesizing AI Analysis

I'm now drafting the five conclusion paragraphs, synthesizing key patterns from the audit. I'm focusing on the anthropomorphic framing and its implications, aiming for a rigorous and concise summary within the word count. I'm prioritizing the analysis, and am working on crafting concise paragraphs. I am making progress.

Synthesizing AI Insights

I'm now synthesizing key patterns from my audit, specifically focusing on the anthropomorphic framing. I'm focusing on "the AI as an Experiencing Subject." The shift from mechanical processes to subjective experience is critical to the other patterns and I'm developing a rigorous and concise summary for the conclusion. My priority is to connect AI, AI's actors and AI's actions. I am finalizing the language.

Analysis Provenance

Run ID: 2026-05-24-probing-persona-dependent-preferences-in-metaphor-o26h61 Raw JSON: 2026-05-24-probing-persona-dependent-preferences-in-metaphor-o26h61.json Framework: Metaphor Analysis v6.5 Schema Version: 3.0 Generated: 2026-05-24T11:55:50.452Z

Discourse Depot © 2025 by TD is licensed under CC BY-NC-SA 4.0