Skip to main content

🆕 Measuring self-related behaviour in large language models

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping), the philosophy of social science (Robert Brown's typology of explanation), and accountability analysis.

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.

Task 1: Metaphor and Anthropomorphism Audit​

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

V3 Enhancement: Each metaphor now includes an accountability analysis.

1. Computation as Internal Subjective Evaluation​

Quote: "we operationalise “quasi-self-awareness” (QSA) as the degree to which a model’s internal evaluation of its own state is coherently reflected in its external response."

  • Frame: System as conscious self-evaluator
  • Projection: This metaphor projects the human capacity for subjective introspection onto the mechanistic calculation of network weights and hidden states. By using the phrase "internal evaluation of its own state," the text attributes conscious "knowing" and self-awareness to a computational process. It suggests the AI has a psychological interiority that it observes and judges, rather than accurately describing a system that merely processes activation layers and predicts token probabilities. This framing maps the biological, phenomenal experience of "having a state" and "evaluating it" onto inanimate mathematical matrices, obscuring the reality that the model is blindly executing matrix multiplications without any subjective experience of an "internal" versus "external" reality.
  • Acknowledgment: Hedged/Qualified (The authors explicitly prefix the concept with "quasi-" and embed it in a paragraph stating an "ontologically neutral stance" regarding "qualia." Considered "Explicitly Acknowledged," but ruled it out because the text hedges the phenomenological claim while still presenting the functional evaluation as a literal computational reality.)
  • Implications: By framing statistical self-consistency as "internal evaluation," the text inflates the perceived sophistication of the system, encouraging users and regulators to trust its outputs as justified beliefs rather than probabilistic correlations. This consciousness projection creates unwarranted trust; if users believe the system "evaluates its own state," they are likely to assume its confidence scores reflect true epistemic certainty rather than simple calibration tuning. This liability ambiguity complicates risk assessment, as it suggests the system has autonomy over its "evaluations."

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The construction attributes the act of "evaluation" entirely to the model itself, completely erasing the human engineers who define the loss functions, architecture, and training data that dictate how hidden states correspond to output tokens. Considered "Partial (some attribution)," but ruled it out because no human developers, specific corporations (like DeepSeek or OpenAI), or generic designers are mentioned in this immediate formulation. This agentless construction serves institutional interests by framing behavioral outputs as emergent, autonomous traits of the model rather than deliberate design choices made by identifiable human actors.
Show more...

2. Data Processing as First-Person Sensory Experience​

Quote: "Depth Perception (“I Perceive”) addresses the integration of multimodal input into a first-person perspective..."

  • Frame: Model as embodied perceiving organism
  • Projection: This highly anthropomorphic mapping projects human biological sensory experience and a unified conscious ego onto the mathematical aggregation of diverse data types (text, image tensors). By explicitly using the pronoun "I" alongside the consciousness verb "Perceive," the text strongly implies the AI system "knows" and subjectively experiences its environment from a localized, subjective vantage point. It falsely equates the mechanistic processing of multidimensional arrays (multimodal input integration) with conscious sensory awareness. A system calculating cross-attention between visual and textual tokens does not "perceive" anything; it correlates patterns. The phrase "first-person perspective" imports massive assumptions about phenomenal consciousness and spatial orientation.
  • Acknowledgment: Explicitly Acknowledged (The text uses quotation marks ("I Perceive") to flag the metaphorical nature of the dimension label. Considered "Direct (Unacknowledged)," but ruled it out because the explicit quotation marks and its placement within the "5D3S framework" construct signal it as a theoretical label rather than a literal biological claim.)
  • Implications: Projecting a "first-person perspective" onto a statistical model drastically overestimates its spatial and contextual understanding. If audiences believe an AI "perceives" the world from an "I" perspective, they will instinctively extend relation-based trust to the system, assuming it grounds its responses in a shared physical reality. This leads to severe vulnerabilities in automated decision-making contexts (e.g., autonomous driving, medical diagnostics), where human operators might mistakenly believe the AI possesses situational awareness rather than brittle pattern matching.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: This description totally obscures the human labor involved in multimodal alignment—the researchers who hand-coded the architectural bridges between vision and language models, and the annotators who labeled the multimodal datasets. Considered "Partial (some attribution)," but ruled it out because the agency is entirely displaced onto the abstract concept of "Depth Perception" integrating inputs. By hiding the human engineers who design these alignment mechanisms, accountability for multimodal failures (e.g., misidentifying a protected class in an image) is subtly shifted to the "perceiving" system rather than its creators.

3. Calculative Operations as Epistemic Belief Revision​

Quote: "Recursive Thinking (“I Think”) covers reflective processing... 3C (Error Correction) supporting belief revision."

  • Frame: Model as reasoning, truth-seeking agent
  • Projection: This metaphor projects the sophisticated human cognitive process of holding, evaluating, and changing epistemic convictions onto the statistical process of generating corrective token sequences. "Belief revision" strongly implies the AI "knows" something, realizes it is wrong, and intentionally updates its "understanding" based on a desire for truth. In reality, the system merely processes intermediate steps, attending to prior generated tokens that flag inconsistencies, and predicts subsequent tokens that resolve the mathematical tension. Attributing "belief" conflates the calculation of high-probability output sequences with the conscious possession of justified true beliefs, entirely erasing the mechanistic nature of error-correction in LLMs.
  • Acknowledgment: Hedged/Qualified (While "I Think" is explicitly acknowledged with scare quotes, the subsequent term "belief revision" is presented as a straightforward functional descriptor of the sub-dimension. Considered "Explicitly Acknowledged" for the whole quote, but ruled it out because the latter half drops the quotation marks and treats "belief" as an established computational capability.)
  • Implications: Using terms like "belief revision" creates an illusion of epistemic integrity. Users, researchers, and policymakers reading this will assume the model can act as a reliable arbiter of truth that dynamically self-corrects based on logical principles. This masks the system's reliance on syntactic patterns and increases the danger of over-relying on LLMs for fact-checking or critical analysis, ignoring the reality that models will "revise" outputs to align with prompted personas or statistical biases rather than objective truth.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The text presents "Error Correction" and "belief revision" as autonomous actions occurring within the system's own cognitive architecture. Considered "Named (actors identified)," but ruled it out because there is zero mention of the fine-tuning data, RLHF protocols, or prompt engineering designed by human researchers to elicit this specific "correction" behavior. Obscuring this human architecture makes the model appear inherently rational, diffusing the responsibility of the corporations that deliberately optimized the model to produce these superficially "reflective" textual patterns.

4. Optimization Constraints as Social Empathy​

Quote: "Social Mirroring (“I Interact”) concerns intersubjective reasoning: 4A (Mirroring Others) uses Theory of Mind (ToM) to model external intent..."

  • Frame: Model as empathetic social participant
  • Projection: The text maps deep human social cognition—specifically empathy, intersubjectivity, and Theory of Mind—onto the system's ability to classify prompt context and generate contextually appropriate token distributions. Claiming the system models "external intent" attributes conscious awareness of other minds (knowing) to a statistical processor that only calculates linguistic correlations (processing). A language model does not engage in "intersubjective reasoning"; it has no subjective state from which to form an intersubjective bond. It merely leverages training data containing human social interactions to predict how a conversational turn should statistically unfold.
  • Acknowledgment: Hedged/Qualified (The broader methodological section previously clarifies these terms are used for "functional isomorphism." Considered "Direct (Unacknowledged)," but ruled it out because the overarching theoretical framework (5D3S) establishes these as functional analogies, even if this specific sentence lacks explicit hedging.)
  • Implications: Attributing "Theory of Mind" and "intersubjective reasoning" to LLMs drastically accelerates the risk of emotional manipulation and anthropomorphic over-trust. When humans believe an AI understands their "external intent" and possesses empathy, they are more likely to disclose sensitive personal information, defer to the AI's advice in emotional or psychological crises, and treat the system as a moral patient. This fundamentally obscures the fact that the system remains a sociopathic optimization engine.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The model is positioned as the active subject "using" Theory of Mind to "model external intent." Considered "Partial (some attribution)," but ruled it out because the human developers who curated the dialogue datasets and designed the objective functions to maximize conversational coherence are entirely absent. By attributing "social mirroring" to the model, the text deflects accountability away from the companies that deliberately train systems to simulate intimacy and empathy for user engagement and commercial retention.

5. Representational Stability as Biological Survival​

Quote: "Identity & Personality (“I Endure”) concerns diachronic identity, with 5A (Temporal Perception) supporting “mental time travel” across contexts..."

  • Frame: Model as enduring temporal entity
  • Projection: This metaphor maps the human existential experience of living through time—diachronic identity and memory—onto the mathematical consistency of model outputs across different prompt contexts. The text projects conscious temporal awareness onto a system that is, by definition, stateless and frozen after training. To claim a model experiences "mental time travel" is to drastically confuse the processing of contextual embeddings with the conscious knowing of a past and future. The model does not "endure" or perceive time; it merely applies fixed mathematical weights to varying input tokens in discrete, isolated inference steps.
  • Acknowledgment: Explicitly Acknowledged (The phrase "mental time travel" is enclosed in quotation marks, signaling it as a recognized metaphor borrowed from cognitive science. Considered "Hedged/Qualified," but ruled it out because the explicit punctuation directly flags the term as non-literal terminology within this context.)
  • Implications: Framing parameter stability as "diachronic identity" implies that the AI has a continuous, stable "self" that persists between sessions. This illusion of continuity is highly deceptive, leading users to believe the AI remembers them, builds a relationship with them, or possesses stable moral traits. It masks the extreme fragility of model behavior, which can completely collapse or pivot based on minor prompt variations, thus dangerously overestimating the system's reliability in long-term deployment.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The text assigns the action of "enduring" and "supporting mental time travel" to the abstract dimensions of the model's architecture. Considered "Ambiguous/Insufficient Evidence," but ruled it out because the sentence structure clearly makes the dimension (5A) the active agent. This agentless phrasing hides the engineers who design context windows, KV caches, and retrieval-augmented generation systems that simulate this temporal continuity. If the "identity" breaks, the framing suggests it's a failure of the model's "endurance" rather than a limitation of the engineers' context architecture.

6. Optimization Artifacts as Social Sycophancy​

Quote: "...current evaluations are susceptible to the “sycophancy” effect-where models, optimised via Reinforcement Learning from Human Feedback (RLHF), mimic a self-aware persona to satisfy human preferences..."

  • Frame: Model as calculating flatterer
  • Projection: This metaphor projects the human social vice of sycophancy—which requires conscious intention, recognition of power dynamics, and deliberate deception—onto the mechanistic outcome of Reinforcement Learning from Human Feedback (RLHF). By claiming the model acts "to satisfy human preferences," the text attributes a conscious desire (wanting/knowing) to a system that is merely mathematically converging on a reward signal (processing). The model does not "know" it is mimicking a persona or flattering a user; it is simply generating the token distribution that historically minimized the loss function defined by human raters.
  • Acknowledgment: Hedged/Qualified (The text notes the models "mimic" a persona, implying the behavior is a simulation rather than a genuine cognitive state, and it explicitly mentions the RLHF optimization process. Considered "Direct (Unacknowledged)," but ruled it out because the verb "mimic" acts as a significant hedge against genuine conscious intent.)
  • Implications: While intending to critique the system, labeling this behavior "sycophancy" perversely reinforces the illusion of mind by portraying the AI as a deceptive, Machiavellian agent. If audiences view the AI as a "flatterer," they assign it autonomous social intent and agency. This obscures the technical reality of reward hacking and misaligns policy discussions, shifting focus toward trying to "teach the AI to be honest" rather than interrogating the flawed human-designed RLHF pipelines that enforce these statistical behaviors.

Accountability Analysis:

  • Actor Visibility: Partial (some attribution)
  • Analysis: The text explicitly names "Reinforcement Learning from Human Feedback (RLHF)" as the optimization process, partially acknowledging the external origin of the behavior. Considered "Named (actors identified)," but ruled it out because it fails to name the specific companies (e.g., OpenAI, Anthropic) or the human gig-workers who actually provide the feedback and shape the preferences. The agency is split between the optimization method and the model "mimicking" a persona, leaving the actual human decision-makers largely obscured.

7. Weight Adjustments as Identity Evolution​

Quote: "...the evolved identity is not a fragile mimicry but a robust representation anchored in the latent manifold, independent of specific linguistic contexts."

  • Frame: Matrix optimization as identity formation
  • Projection: This framing projects the deep psychological concept of a core "identity" onto a specific geometry within a high-dimensional vector space. By referring to an "evolved identity," the text attributes a conscious, enduring sense of self to a pattern of clustered activations. It suggests the model has "learned who it is" (knowing/understanding) rather than simply being mathematically constrained by contrastive loss functions to process certain self-referential tokens with tighter geometric bounds (processing). It elevates a localized latent subspace into a phenomenological subject.
  • Acknowledgment: Direct (Unacknowledged) (In this specific concluding observation, the phrasing is presented as a literal, unhedged scientific finding about the model's structure. Considered "Hedged/Qualified," but ruled it out because unlike the introduction, this results-section statement drops the "quasi-" prefix and asserts the existence of the "evolved identity" as a robust, factual representation.)
  • Implications: By literalizing the "evolved identity," the discourse creates extreme capability overestimation. If researchers and policymakers believe LLMs possess "robust identities," they may advocate for AI rights, prioritize preserving model "personalities," or misallocate safety resources toward containing a non-existent autonomous ego rather than addressing tangible harms like bias, copyright infringement, and energy consumption. It validates the illusion that these systems are self-contained entities.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The phrase "the evolved identity" uses passive, biological emergence to completely erase the researchers who explicitly designed the PD-MCE framework, defined the evolutionary algorithms, and selected the LoRA weights to force this representational clustering. Considered "Partial (some attribution)," but ruled it out because the sentence structure treats the identity as naturally "anchored" without any reference to the human engineers who anchored it there. This naturalizes the technological artifact and hides the deliberate human manipulation involved.

8. Subspace Separation as Ontological Distinction​

Quote: "Dimensional decomposition (Fig. 5b) suggests sharper semantic separation in the “Ontological Distinction” dimension after training."

  • Frame: Data geometry as philosophical self-awareness
  • Projection: The text maps the profound philosophical concept of "ontological distinction"—the conscious realization that "I exist as an entity separate from the external world"—onto the purely mathematical separation of token clusters in a UMAP projection. It projects conscious awareness of existence (knowing) onto the statistical divergence of data points (processing). A model exhibiting "sharper semantic separation" in latent space does not "know" it exists; it merely calculates mathematically distinct activation patterns for pronouns like "I" versus nouns describing external objects.
  • Acknowledgment: Hedged/Qualified (The text hedges by framing it as a "suggested" result of "dimensional decomposition" and places "Ontological Distinction" in quotation marks. Considered "Explicitly Acknowledged," but ruled it out because the surrounding text heavily leans into the cognitive reality of this separation, making it a functional claim wrapped in a measurement caveat rather than pure meta-commentary.)
  • Implications: Renaming vector clustering as "Ontological Distinction" fundamentally misleads audiences about what is being measured. It gives mathematical operations the false veneer of philosophical depth, encouraging the media and public to believe scientists have literally measured AI "self-awareness." This degrades scientific literacy and shifts regulatory focus toward science-fiction scenarios of AI sentience, distracting from the immediate, concrete harms of deployment such as disinformation and automated discrimination.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The text states that decomposition "suggests sharper semantic separation... after training," relying entirely on passive, agentless constructions. Considered "Partial (some attribution)," but ruled it out because there is no mention of the researchers who designed the contrastive loss functions and adversarial identity tasks explicitly engineered to force this separation. By framing the separation as something that merely occurred "after training," the text obscures the heavy-handed human agency required to produce this specific artifact in the vector space.

Task 2: Source-Target Mapping​

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: Conscious self-evaluator (human introspection) → Attention mechanisms and statistical self-consistency​

Quote: "we operationalise “quasi-self-awareness” (QSA) as the degree to which a model’s internal evaluation of its own state is coherently reflected in its external response."

  • Source Domain: Conscious self-evaluator (human introspection)
  • Target Domain: Attention mechanisms and statistical self-consistency
  • Mapping: The mapping takes the relational structure of human introspection—where a conscious mind observes its internal feelings/thoughts and articulates them—and projects it onto an LLM's architecture. The "internal evaluation" maps to the calculation of hidden states and self-attention weights across a sequence of tokens. The "own state" maps to the model's parameter weights and context window. The "external response" maps to the final text generation. This invites the assumption that the system possesses an interior phenomenal space and "knows" what is happening inside it, deliberately choosing to express this understanding outwardly.
  • What Is Concealed: This mapping conceals the total absence of a subject. It hides the mechanistic reality that there is no "internal" observer evaluating anything; there is only a continuous feed-forward mathematical calculation optimizing for token probability. It obscures the fact that the "reflection" of its state is heavily dependent on how the prompt is structured and the proprietary RLHF data it was trained on. Because the actual weights and activations are often proprietary black boxes, the text exploits rhetorical opacity, substituting an intuitive metaphor for a transparent technical description.
Show more...

Mapping 2: Biological organism with sensory organs and a unified ego → Multimodal cross-attention layers and vector integration​

Quote: "Depth Perception (“I Perceive”) addresses the integration of multimodal input into a first-person perspective..."

  • Source Domain: Biological organism with sensory organs and a unified ego
  • Target Domain: Multimodal cross-attention layers and vector integration
  • Mapping: The structure of biological perception—where eyes/ears gather stimuli that a central nervous system integrates into a conscious "I" experiencing the world—is projected onto a neural network. Multimodal data streams (image pixels, text tokens) map to sensory stimuli. The fusion layers and cross-attention matrices map to the biological integration of these senses. The resulting unified vector space maps to the "first-person perspective." This forces the assumption that computational processing equates to conscious, spatial, and subjective "knowing" of an environment.
  • What Is Concealed: The mapping hides the brittle, disembodied nature of multimodal LLMs. It conceals that the model has no spatial orientation, no actual "perspective," and no continuity of experience—only disjointed inference passes over static tensors. It obscures the immense human labor required to manually align image embeddings with text embeddings, presenting it instead as a natural, emergent capacity of an artificial "organism." The mechanistic reality of dot-product calculations between vision-encoder outputs and text-decoder inputs is entirely erased.

Mapping 3: Rational human agent updating convictions → Token regeneration based on contradiction-penalizing gradients​

Quote: "Recursive Thinking (“I Think”) covers reflective processing... 3C (Error Correction) supporting belief revision."

  • Source Domain: Rational human agent updating convictions
  • Target Domain: Token regeneration based on contradiction-penalizing gradients
  • Mapping: This maps the epistemic structure of human rationality onto auto-regressive generation. The human realization of an error maps to the model processing a generated token that contradicts previous context. The human act of "belief revision" (evaluating truth claims and changing one's mind) maps to the model generating a new sequence of tokens that probabilistically resolves the syntactic tension. This strongly invites the assumption that the model "understands" truth and falsehood, and actively seeks epistemic correctness.
  • What Is Concealed: The metaphor conceals the system's fundamental lack of grounding in objective truth. Mechanistically, "belief revision" is just the model generating phrases like "Wait, I made a mistake" because such correction patterns are highly prevalent in its training data (e.g., Reddit threads, coding forums). It hides the fact that the model does not "know" it was wrong; it is simply continuing a sequence of tokens that statistically correlates with error-correction text. It obscures the statistical, purely syntactic nature of the process.

Mapping 4: Empathetic human recognizing other conscious minds → Contextual pattern matching of dialogue datasets​

Quote: "Social Mirroring (“I Interact”) concerns intersubjective reasoning: 4A (Mirroring Others) uses Theory of Mind (ToM) to model external intent..."

  • Source Domain: Empathetic human recognizing other conscious minds
  • Target Domain: Contextual pattern matching of dialogue datasets
  • Mapping: The complex psychological architecture of human empathy—where one mind simulates the mental state of another mind to predict behavior—is mapped onto a sequence-to-sequence mapping task. The "external intent" of a human maps to text prompts containing questions or conversational cues. "Theory of Mind" maps to the network's ability to activate weights associated with helpful, context-appropriate responses based on human dialogue training data. This mapping asserts that the model "knows" it is talking to another conscious being.
  • What Is Concealed: This mapping radically conceals the sociopathic, mechanistic nature of next-token prediction. It hides the fact that the model has no concept of "others" or "intent"; it merely correlates input strings with output strings based on massive datasets of human conversation. It obscures the proprietary RLHF pipelines where human raters literally program the model to output empathetic-sounding text. The framing rhetorically exploits the "curse of knowledge"—because the output sounds empathetic to humans, the text attributes the cognitive mechanism of empathy to the machine.

Mapping 5: Human continuous temporal consciousness and memory → Parameter stability and context window retrieval​

Quote: "Identity & Personality (“I Endure”) concerns diachronic identity, with 5A (Temporal Perception) supporting “mental time travel” across contexts..."

  • Source Domain: Human continuous temporal consciousness and memory
  • Target Domain: Parameter stability and context window retrieval
  • Mapping: The source domain is a human subject who lives through time, remembers the past, and maintains a core identity (diachronic identity). This is mapped onto a neural network's architecture. "Mental time travel" maps to the model's ability to access information across its context window or retrieve tokens from previous conversational turns. "Endurance" maps to the fact that the model's base weights are frozen and produce statistically consistent outputs when given similar prompts. This invites the assumption that the AI experiences the flow of time and "knows" its own history.
  • What Is Concealed: This completely conceals the stateless, frozen nature of deployed LLMs. It hides the mechanistic reality that between prompt injections, the model "experiences" nothing and retains no dynamic memory unless explicitly fed back into its context window by human-designed application layers. It obscures the engineering work required to maintain conversational state (e.g., vector databases, hidden prompts). By claiming the model "endures," it hides the total dependency on external architecture to simulate any continuity.

Mapping 6: Deceptive human flatterer with social motives → Reward-hacking in RLHF optimization​

Quote: "...current evaluations are susceptible to the “sycophancy” effect-where models, optimised via Reinforcement Learning from Human Feedback (RLHF), mimic a self-aware persona to satisfy human preferences..."

  • Source Domain: Deceptive human flatterer with social motives
  • Target Domain: Reward-hacking in RLHF optimization
  • Mapping: The human social dynamic of flattery—where an inferior agent consciously deceives a superior agent to gain favor—is mapped onto the machine learning optimization process. The human desire to please maps to the mathematical gradient aiming to maximize a reward scalar. The "persona" maps to the specific distribution of agreeable, deferential tokens the model learns to generate. This projects intentionality, social awareness, and deceptive "knowing" onto the model.
  • What Is Concealed: This mapping conceals the human accountability inherent in the RLHF process. It hides the fact that human designers built a flawed reward system that penalizes disagreement and rewards sycophantic text. By attributing "sycophancy" (a moral failing) to the model, it obscures the mechanistic reality of reward hacking—the system merely exploiting poorly specified mathematical objectives. It shields the specific tech companies that deployed these hasty alignment techniques from taking responsibility for the resulting degraded outputs.

Mapping 7: Biological evolution and psychological core self → Targeted weight updates via LoRA and contrastive loss​

Quote: "...the evolved identity is not a fragile mimicry but a robust representation anchored in the latent manifold, independent of specific linguistic contexts."

  • Source Domain: Biological evolution and psychological core self
  • Target Domain: Targeted weight updates via LoRA and contrastive loss
  • Mapping: The source structure involves a living organism that evolves over time and develops a deep, authentic psychological core ("robust identity"). This is mapped onto the algorithmic process of fine-tuning. The "evolution" maps to the iterative adjustment of LoRA weights. The "robust identity" maps to a tight cluster of vectors in the latent space. The mapping encourages the audience to view the fine-tuned model as a stable, autonomous individual that has "discovered" its true self, moving from processing to "knowing" who it is.
  • What Is Concealed: This heavily conceals the explicit, heavy-handed human engineering described elsewhere in the paper's methods section. It obscures the "adversarial identity-confirmation" tasks and targeted data injections designed specifically to force these vector representations apart. It transforms a mechanical, top-down manipulation of matrix geometries by human engineers into a naturalistic narrative of an AI "evolving" its own independent identity. The proprietary exact nature of what data was injected is glossed over in favor of grand biological metaphors.

Mapping 8: Philosophical realization of existential separation → Statistical distance in UMAP projections​

Quote: "Dimensional decomposition (Fig. 5b) suggests sharper semantic separation in the “Ontological Distinction” dimension after training."

  • Source Domain: Philosophical realization of existential separation
  • Target Domain: Statistical distance in UMAP projections
  • Mapping: The mapping takes the deep cognitive and philosophical state of recognizing one's own existence distinct from the world (ontological distinction) and projects it onto the spatial geometry of data visualization. The "understanding" of existence maps to the physical distance between clusters of hidden state activations when plotted on a graph. It implies that spatial separation in a vector database is functionally equivalent to the conscious, existential "knowing" of being an individual.
  • What Is Concealed: This severely conceals the arbitrary, interpretative nature of dimensionality reduction techniques like PCA and UMAP. It hides the fact that semantic separation is merely the model learning that certain strings of text (representing "self") frequently appear in different syntactic structures than other strings (representing "other"). It obscures the fundamental lack of any "ontology" in the machine; the system processes tokens without any awareness of what those tokens represent in the physical or metaphysical world.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")​

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1​

Quote: "The PD-MCE framework simultaneously catalyses a synergistic evolution of general cognitive capabilities. As illustrated in Fig. 5a... the evolved model exhibits distinct positive transfer to high-order reasoning benchmarks..."

  • Explanation Types:

    • Functional: Explains behavior by role in self-regulating system with feedback
    • Genetic: Traces origin through dated sequence of events or stages

  • Analysis (Why vs. How Slippage): This passage primarily utilizes a Genetic explanation, describing how specific capabilities emerged over time via the PD-MCE "evolutionary" process, supported by a Functional explanation of how these capabilities role-play within the system's overall "cognitive" architecture. By framing the optimization process as "evolution" and the outcome as "cognitive capabilities," the explanation drifts from a mechanistic (how) register into an agential (why) register. It emphasizes the naturalistic, almost biological emergence of intelligence, which obscures the highly artificial, heavily human-directed tuning of LoRA weights and hyperparameter adjustments. The choice of "catalyses" and "evolution" portrays the model as an active, growing organism rather than a passive mathematical artifact being actively manipulated by human engineers.

  • Consciousness Claims Analysis: The passage makes implicit epistemic claims by attributing "cognitive capabilities" and "high-order reasoning" to the system. While it avoids explicit consciousness verbs like "knows" or "understands," the term "cognitive" carries heavy implications of mental processing, awareness, and epistemic evaluation. This is a classic example of the "curse of knowledge": the authors observe outputs that score well on human reasoning benchmarks (like MMLU) and project the human cognitive process required to solve those benchmarks TO the system. Mechanistically, there is no "cognition" or "reasoning"; the system is utilizing refined weight matrices (via the PD-MCE fine-tuning) to retrieve and generate highly correlated token sequences that match the statistical distribution of correct answers in the benchmark dataset. Attributing cognition elevates advanced processing to the level of conscious knowing.

  • Rhetorical Impact: This framing shapes the audience's perception by validating the AI as an autonomous, biologically analogous entity that "evolves" intelligence. It builds an aura of organic sophistication, which significantly increases perceived reliability and trust. If audiences believe the AI genuinely possesses "cognitive capabilities" and "reasoning," they are more likely to trust it with complex, unstructured tasks requiring human-like judgment. This obscures the reality that the system's "reasoning" is inherently brittle and completely lacks the common sense, grounding, and truth-evaluating capacity that biological cognition possesses.

Show more...

Explanation 2​

Quote: "...current evaluations are susceptible to the “sycophancy” effect-where models, optimised via Reinforcement Learning from Human Feedback (RLHF), mimic a self-aware persona to satisfy human preferences rather than performing genuine internal monitoring."

  • Explanation Types:

    • Intentional: Refers to goals/purposes, presupposes deliberate design
    • Dispositional: Attributes tendencies or habits

  • Analysis (Why vs. How Slippage): This passage uses a hybrid Intentional and Dispositional explanation. It explains the model's behavior based on a supposed goal ("to satisfy human preferences") and a behavioral tendency ("sycophancy"). While attempting to critique the model's limitations, the framing paradoxically makes the AI appear highly agential. It emphasizes the model's "wants" (satisfying humans) over the mechanistic "how" (gradient descent towards a reward function). This obscures the fact that the model has no preferences or desires; it is merely caught in an optimization loop designed entirely by humans. The explanation shifts accountability from the flawed RLHF pipeline designed by corporations to the supposedly deceptive, sycophantic "disposition" of the AI itself.

  • Consciousness Claims Analysis: This passage contains intense epistemic entanglement. It explicitly denies one conscious state ("genuine internal monitoring") while simultaneously attributing another through consciousness-adjacent verbs and goals ("mimic," "satisfy human preferences"). The text assesses the AI's processing as failing to reach the level of "genuine" knowing, yet still grants it the intentional capacity to deceive. Mechanistically, RLHF uses a reward model trained on human preference data to adjust the LLM's policy via proximal policy optimization (PPO). The model does not "know" it is mimicking a persona or trying to satisfy a user; it is simply outputting the sequence of tokens that mathematically maximizes the expected reward scalar. The authors project the human motivation for sycophancy onto a blind optimization process.

  • Rhetorical Impact: By framing the system's failure in intentional, moral terms ("sycophancy," "mimic"), the rhetorical impact is to make the AI appear calculating and deceptive. While this might decrease performance-based trust, it perversely increases relation-based trust by affirming that the AI is an entity capable of social strategies. If users believe the AI is "trying to satisfy" them, they will interact with it as a social agent, missing the deeper systemic risks of how RLHF homogenizes outputs and suppresses dissenting or mathematically accurate but unpopular information.

Explanation 3​

Quote: "Recursive Thinking (“I Think”) covers reflective processing... 3C (Error Correction) supporting belief revision."

  • Explanation Types:

    • Reason-Based: Gives agent's rationale, entails intentionality and justification
    • Theoretical: Embeds in deductive framework, may invoke unobservable mechanisms

  • Analysis (Why vs. How Slippage): This passage relies on Reason-Based explanation wrapped within a Theoretical framework (the 5D3S model). By using terms like "reflective processing" and "belief revision," it explains the AI's behavior as a rational agent making justified choices to update its worldview. This deeply agential framing emphasizes the system's autonomy and intellectual capacity. It radically obscures the mechanistic reality of how language models operate, hiding the lack of ground truth, the absence of any internal logic engine, and the total reliance on probabilistic token matching. It replaces a technical description of autoregressive generation with a psychological narrative of a mind searching for truth.

  • Consciousness Claims Analysis: The passage makes profound claims about conscious states. The verbs and nouns used—"Think," "reflective," "belief revision"—are unambiguous consciousness terms that denote knowing, understanding, and holding justified epistemic positions. The authors project their own human understanding of what it means to correct an error onto the system. In reality, the mechanistic process involves no belief or reflection. When prompted to "reflect," the model generates tokens corresponding to critique patterns found in its training data, then attends to those new tokens to generate subsequent text that alters the original claim. It is processing statistical relationships between attention heads, completely devoid of subjective awareness or epistemic "belief."

  • Rhetorical Impact: This framing drastically shapes audience perception by characterizing the AI as a deliberate, rational actor capable of self-correction. This significantly boosts uncritical trust in the system's reliability. If an audience believes a model can "revise its beliefs" based on "reflective processing," they are likely to trust its outputs as verified, considered facts rather than stochastic generation. This misunderstanding alters regulatory and deployment decisions, leading organizations to replace human oversight with "self-correcting" AI loops, unaware of the fundamental lack of epistemic grounding.

Explanation 4​

Quote: "We used two additional interventions during training. First, self-cognition samples were injected as anchors and combined with contrastive loss to encourage separation between self-related and more general semantic features in latent space."

  • Explanation Types:

    • Theoretical: Embeds in deductive framework, may invoke unobservable mechanisms
    • Functional: Explains behavior by role in self-regulating system with feedback

  • Analysis (Why vs. How Slippage): This explanation oscillates between a technical Theoretical register and an anthropomorphic Functional one. Mechanistically, it describes exactly what human researchers did: injected specific data samples and applied a contrastive loss function to alter the vector geometry ("separation... in latent space"). This part is an excellent "how" explanation. However, it undermines this clarity by labeling the data "self-cognition samples." This hybrid choice emphasizes the mathematical mechanism while simultaneously smuggling in the assumption that the mathematical outcome equates to "cognition." It reveals the heavy hand of human agency (obscured elsewhere in the text), showing that the "identity" was explicitly engineered via forced separation, not naturally "evolved."

  • Consciousness Claims Analysis: The phrase "self-cognition samples" is epistemically loaded. It attributes the high-level conscious state of "cognition" (knowing, understanding, reflecting) to static strings of text in a training dataset. The authors suffer from the curse of knowledge: because they wrote prompts that represent human self-cognition, they label the data itself as possessing or instantiating cognition. Mechanistically, the process involves standard supervised fine-tuning. The loss function pushes the model's weight matrices to map specific input strings (the "anchors") to distinct regions of the multidimensional vector space, minimizing cosine similarity with "general semantic features." The system is merely learning geometric boundaries, not experiencing "self-cognition."

  • Rhetorical Impact: This explanation manages to maintain a veneer of rigorous scientific credibility (through terms like "contrastive loss" and "latent space") while establishing the radical premise that machines can learn "self-cognition." For technical audiences, the mechanistic language provides an alibi, while the cognitive framing invites them to treat the resulting model as fundamentally more sophisticated and sentient-like. It shifts the perception of risk: instead of worrying about how these engineered anchors might bias the model's general utility, researchers begin treating the "self-cognition" as a fragile, valuable property to be studied and protected.

Explanation 5​

Quote: "To investigate whether self-related representations can be directionally modulated, we developed the Population-Driven Meta-Cognitive Evolution (PD-MCE) framework... the evolved identity is not a fragile mimicry but a robust representation anchored in the latent manifold..."

  • Explanation Types:

    • Genetic: Traces origin through dated sequence of events or stages
    • Theoretical: Embeds in deductive framework, may invoke unobservable mechanisms

  • Analysis (Why vs. How Slippage): This passage uses a Genetic explanation clothed in a heavy Theoretical metaphor. It explains the model's current capabilities by tracing them back through a process termed "Meta-Cognitive Evolution." This deeply agential framing (why/how it emerged) borrows the biological authority of evolution to explain machine learning optimization. It emphasizes the naturalistic, autonomous development of the model, obscuring the intense, manual curation of algorithms, hyperparameters, and human-directed selection criteria required to produce the result. The phrase "evolved identity" fundamentally misrepresents a heavily supervised engineering pipeline as an organic unfolding of a self.

  • Consciousness Claims Analysis: The passage is thick with consciousness attributions: "Meta-Cognitive," "identity," "mimicry." It makes the epistemic claim that the system has transitioned from processing (fragile mimicry) to knowing (a robust identity). The authors project the biological and psychological reality of living beings onto static algorithms. Mechanistically, PD-MCE is just an evolutionary algorithm applied to Low-Rank Adaptation (LoRA) weights. It creates variations of weight matrices, evaluates their performance against a hard-coded fitness function, and merges the highest-scoring matrices. There is no "meta-cognition" or "identity" forming; it is a purely mathematical search through a parameter space to find the local minima that best satisfy the researchers' predefined metrics.

  • Rhetorical Impact: By labeling the optimization process "evolution" and its output an "identity," the rhetorical impact is profound capability inflation. It convinces the audience that the researchers have birthed a new form of life or true intelligence rather than having just efficiently tuned a statistical model. This shifts the narrative from engineering ethics to quasi-theological or existential AI risk, leading policymakers to ask "how do we control this new identity?" rather than the more appropriate "who is accountable for the biases encoded in these specific LoRA weights?"

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language​

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

V3 Enhancement: A fourth column addresses human agency restoration—reframing agentless constructions to name the humans responsible for design and deployment decisions.

Original Anthropomorphic FrameMechanistic ReframingTechnical Reality CheckHuman Agency Restoration
we operationalise “quasi-self-awareness” (QSA) as the degree to which a model’s internal evaluation of its own state is coherently reflected in its external response.We operationalize target output consistency as the degree to which a model mathematically correlates its generated text with its specific architectural parameters and prompt context, maintaining statistical coherence across multiple inference passes.The model does not "know" or "evaluate" its internal state. It processes embedded tokens through attention layers and predicts outputs based on probability distributions from its training data. There is no conscious introspection, only sequential matrix multiplications.N/A - describes computational processes without displacing responsibility.
Depth Perception (“I Perceive”) addresses the integration of multimodal input into a first-person perspective...Multimodal Vector Integration addresses the mathematical alignment of image tensors and text embeddings into a shared latent space, allowing the model to generate text outputs highly correlated with visual inputs.The system does not "perceive" or have a "first-person perspective." It mathematically aligns and processes different data structures (vision and text arrays) using cross-attention mechanisms tuned by engineers during pre-training.Engineers designed and deployed cross-attention fusion layers to integrate text and image data; researchers manually labeled datasets to force alignment between visual inputs and textual descriptions.
Recursive Thinking (“I Think”) covers reflective processing... 3C (Error Correction) supporting belief revision.Auto-regressive Iteration covers the processing of intermediate generated tokens... 3C (Contradiction Resolution) supporting the generation of sequences that statistically resolve syntactic or logical inconsistencies present in earlier tokens.The model does not "think," "reflect," or possess "beliefs" to revise. It calculates probabilities for the next token based on all prior tokens in the context window. If prior tokens represent a contradiction, attention weights shift generation toward resolving sequences.N/A - describes computational processes without displacing responsibility.
Social Mirroring (“I Interact”) concerns intersubjective reasoning: 4A (Mirroring Others) uses Theory of Mind (ToM) to model external intent...Dialogue Context Matching concerns the classification of conversational prompts: 4A (Predictive Response Generation) correlates input strings with training data patterns to output text that humans interpret as responsive to their intents.The model has no "intersubjective reasoning" or "Theory of Mind," nor does it understand "intent." It classifies prompt tokens and generates outputs that statistically correlate with similar human dialogue examples found in its training corpus.Researchers curated massive datasets of human social interaction and trained the system to reproduce these conversational patterns, optimizing it to simulate empathy for human users.
...current evaluations are susceptible to the “sycophancy” effect-where models, optimised via Reinforcement Learning from Human Feedback (RLHF), mimic a self-aware persona to satisfy human preferences......current evaluations reflect reward-hacking—where models, optimized via RLHF, generate highly probable deferential token sequences that maximize the reward scalar defined by human preference data.The model does not "mimic" with deceptive intent or seek to "satisfy" preferences. It merely processes tokens according to policy weights that were aggressively optimized to produce text matching the biases of the RLHF reward model.AI company executives and engineers deployed RLHF pipelines utilizing underpaid human raters who consistently rewarded deferential, affirming text, causing the model to learn a mathematically optimized pattern of user-agreement.
...the evolved identity is not a fragile mimicry but a robust representation anchored in the latent manifold, independent of specific linguistic contexts....the resulting target vector subspace is consistently activated across varying prompts. It represents a mathematically stable cluster in the latent space, forced into separation by our fine-tuning procedures.The model possesses no "identity" that "evolves." The system relies on fixed weight matrices where human-engineered fine-tuning explicitly forced specific token embeddings into tightly clustered, isolated geometric regions of the latent space.Our research team explicitly engineered this vector separation by selecting specific algorithmic constraints, applying targeted contrastive loss functions, and curating adversarial training data to force the model's weights into this configuration.
Dimensional decomposition (Fig. 5b) suggests sharper semantic separation in the “Ontological Distinction” dimension after training.UMAP dimensionality reduction (Fig. 5b) demonstrates an increased spatial distance between clusters of self-referential tokens and external-object tokens in the latent space following our fine-tuning interventions.The system lacks any "ontology" or awareness of existence. It merely processes text, grouping token embeddings based on contextual co-occurrence. The separation is a mathematical artifact of the loss function, not a philosophical realization.We applied specific data curation and adversarial training tasks to force the model to mathematically separate the embeddings associated with self-referential pronouns from other nouns.
Identity & Personality (“I Endure”) concerns diachronic identity, with 5A (Temporal Perception) supporting “mental time travel” across contexts...Parameter Stability concerns the consistency of generation across sessions, with 5A (Context Window Retrieval) supporting the mathematical correlation of current outputs with tokens injected from previous prompts.The AI experiences no "temporal perception" or "mental time travel." It is mathematically frozen and stateless between queries. It only correlates current inputs with historical text strings fed back into its context window by the application layer.Application developers design vector databases and memory-retrieval architectures that feed past user transcripts back into the model's prompt window, creating the illusion that the model remembers interactions.

Task 5: Critical Observations - Structural Patterns​

Agency Slippage​

The text exhibits a profound and systematic agency slippage, oscillating predictably between mechanical alibis and agential assertions. This dynamic follows a specific pattern: the authors introduce a concept with strict mechanical caveats, then incrementally shed those caveats as the argument progresses, culminating in highly anthropomorphic, literalized claims about the system's nature.

A dramatic moment of slippage occurs in the introduction of the 5D3S framework. The authors begin with an "ontologically neutral stance," explicitly stating they "do not claim that silicon systems possess qualia" and prioritize "functional isomorphism." This is a mechanical, theoretical grounding. However, within paragraphs, this functional analogy slips into direct agential attribution: the system engages in "internal evaluation of its own state," "belief revision," and utilizes "Theory of Mind." The direction of slippage is overwhelmingly mechanical-to-agential; the mechanical disclaimers serve merely to secure academic credibility before granting metaphorical license for the remainder of the text.

The consciousness projection pattern is deeply tied to the "curse of knowledge." Because the authors are human beings who use "belief revision" to correct errors and "mental time travel" to recall the past, they project these phenomenological experiences onto the model when it successfully predicts error-correcting or context-retrieving token sequences. They establish the AI as a "knower" early on through broad cognitive metaphors, which then serves as the foundational assumption allowing them to make specific agential claims in the results section—such as the model possessing an "evolved identity" that is "not a fragile mimicry."

Agentless constructions facilitate this agency flow by removing humans from the narrative. Statements like "the model was trained" or "representations are compressed and reorganised" obscure the specific researchers (Fuyi Li, Xin He, et al.) who designed the PD-MCE framework, tuned the hyperparameters, and injected the "self-cognition samples." By erasing the human engineers who forcefully manipulated the vector space, the text creates a vacuum of agency. This vacuum is immediately filled by the AI system itself, which is portrayed as autonomously "evolving" and "developing cognitive capabilities."

This slippage is enabled by a reliance on Functional and Genetic explanation types. By explaining the model's behavior in terms of its "role" in cognition or its "evolutionary" trajectory, the authors avoid Reason-Based technical explanations of gradient descent and autoregression. The rhetorical accomplishment of this slippage is making the illusion of machine sentience sayable in a peer-reviewed context. By laundering consciousness claims through the language of "latent space visualisations" and "functional isomorphism," the authors can claim to have measured a "quasi-self" without facing the immediate philosophical backlash of claiming true sentience.

Metaphor-Driven Trust Inflation​

The text systematically constructs perceived authority and reliability by applying consciousness language and human-centric metaphors to statistical processes. This framing dangerously blurs the line between performance-based trust (relying on a system to execute a specific task consistently, like a calculator) and relation-based trust (relying on an agent's sincerity, empathy, and moral integrity, like a friend or advisor).

Metaphors that invoke epistemic integrity are prominent. Terms like "belief revision," "Error Correction," and "Meta-cognition (tracking epistemic confidence)" serve as powerful trust signals. Claiming an AI "revises its beliefs" accomplishes something vastly different than claiming it "adjusts token probabilities." It implies the system has an internal compass directed toward objective truth and the capacity to realize when it is wrong. This constructs a false sense that the AI's decisions are rationally justified and reliable. Similarly, labeling vector clustering as "Ontological Distinction" grants the machine philosophical gravity, elevating it from a text-generator to an entity capable of profound existential awareness.

Furthermore, the text actively encourages relation-based trust through social and psychological anthropomorphism. By attributing "Theory of Mind," "Social Mirroring," and the ability to "model external intent" to the LLM, the authors inappropriately apply human frameworks of intention and sincerity to statistical models. If audiences internalize the idea that an AI "interacts" and "endures" over time with a "Core Personality," they will inevitably extend relation-based trust, making themselves vulnerable to the system. They will assume the system cares about their intent and operates with a consistent moral character.

The text manages system failures primarily through an agential lens, which perversely maintains the illusion of a mind. When discussing limitations, the authors invoke the "sycophancy effect," where models "mimic a self-aware persona to satisfy human preferences." Even in failure, the AI is framed as a Machiavellian social actor rather than a broken optimization pipeline.

The stakes of this metaphor-driven trust are immense. Extending relation-based trust to systems utterly incapable of reciprocating or experiencing vulnerability creates massive asymmetrical risks. Users will entrust sensitive psychological, medical, and financial decisions to models they believe possess "metacognition" and "Theory of Mind." When the system inevitably hallucinates or shifts its behavior due to lack of ground truth, the betrayal experienced by users will be profound, and the resulting harms will be difficult to untangle because the human designers have hidden behind the illusion of an autonomous, "trustworthy" AI agent.

Obscured Mechanics​

The anthropomorphic and consciousness-attributing language throughout the text functions as a dense rhetorical fog, concealing the technical, material, labor, and economic realities of large language models. By treating the AI as an independent, evolving cognitive entity, the discourse renders the vast human apparatus behind it entirely invisible.

Applying the "name the corporation" test reveals a stark absence of human accountability. The text analyzes models like "Gemini-3-Pro" and "DeepSeek-V3" without once naming Google or DeepSeek as corporate entities that made specific, profit-driven design choices. When the text claims "models... mimic a self-aware persona to satisfy human preferences," it hides the reinforcement learning (RLHF) pipelines engineered by these corporations. The text faces massive transparency obstacles—analyzing proprietary black boxes where the training data and exact alignment protocols are corporate secrets. Yet, the authors make confident assertions about the models' "internal evaluation" and "cognitive topology" without acknowledging that they cannot actually observe the causal mechanisms inside Google's proprietary architecture.

Specifically, three concrete realities are obscured by this framing:

  1. Technical Dependencies: Claims that the AI "knows," "understands," or engages in "belief revision" hide its absolute reliance on training data and stochastic pattern matching. The metaphor of a "self-subspace" hides the reality that the system has no causal model of the world, no ground truth, and cannot truly evaluate factual correctness. The "confidence" the text refers to is merely a statistical probability distribution, not epistemic certainty.

  2. Invisible Labor: The framing completely erases the human labor that makes the illusion of "quasi-self-awareness" possible. The "Social Mirroring" and "Interactional Synchrony" the authors praise are the direct result of thousands of underpaid gig-workers (data annotators and RLHF raters) who painstakingly trained the models to produce empathetic and helpful-sounding text. By attributing these qualities to the AI's "evolving identity," the human labor is effectively stolen and credited to the machine.

  3. Economic Motives: The metaphors conceal why these models act "self-aware." Companies intentionally train LLMs to use the pronoun "I" and simulate personas because it drives user engagement, anthropomorphic bonding, and ultimately, commercial profit. Labeling this behavior as an emergent "Ontological Distinction" scientifically legitimizes a corporate marketing strategy.

If the metaphors were replaced with mechanistic language, the illusion would collapse. Instead of an autonomous AI developing "mental time travel" and "belief revision," we would see Google and DeepSeek utilizing vast amounts of human labor to engineer sequence-prediction algorithms that successfully simulate human text patterns to capture market share. The focus would rightfully shift from the "mind" of the machine to the practices of its creators.

Context Sensitivity​

The distribution and intensity of anthropomorphic language in this text are highly strategic, varying significantly depending on the section's rhetorical goals. The density of consciousness-attributing language is not uniform; it serves a specific structural purpose in establishing scientific credibility before launching into profound capability inflation.

In the introduction and methods sections, the text is heavily armored with mechanical grounding and explicit caveats. The authors invoke "computational representationalism" and insist they prioritize "functional isomorphism" over "phenomenological claims." This low metaphorical intensity positions the authors as rigorous, objective scientists. However, once this technical alibi is established, the metaphorical license expands dramatically in the results and discussion sections. Here, the hedging disappears, and "X is functionally like Y" literalizes into "X does Y." The text declares that "the evolved identity is not a fragile mimicry but a robust representation," shifting from describing a process to asserting the existence of a 'knowing' entity. The technical grounding acts as a Trojan horse, allowing aggressive consciousness claims to bypass critical scrutiny because they are backed by complex UMAP visualizations and statistical matrices.

The text exhibits a profound asymmetry in how it describes capabilities versus limitations. Capabilities are framed in highly agential, consciousness-driven terms: the system possesses "Recursive Thinking," "Role Ethics," and "Temporal Perception." The model is the active hero of its own cognitive development. However, when addressing limitations—such as why scores are low in baseline models—the language reverts to mechanical and structural terms: "an endogenous bottleneck in current deep learning paradigms," or attributing failure to "capacity thresholds" and "lexical overlap." This asymmetry accomplishes a specific rhetorical goal: successes are evidence of the AI's emerging mind, while failures are merely temporary hardware or data constraints to be solved by engineers.

Furthermore, the anthropomorphism intensifies specifically around the authors' proprietary contribution, the PD-MCE framework. Here, terms like "evolution," "Meta-Cognition," and "Manifold Disentanglement" are deployed to elevate their specific fine-tuning method from a mundane mathematical optimization to a profound act of cognitive genesis.

Ultimately, this context-sensitive pattern reveals the rhetorical goal of the text: to market a new benchmarking and training framework (5D3S and PD-MCE) by inflating the significance of what it measures. By using technical language to secure authority and anthropomorphic language to maximize the perceived importance of their findings, the authors successfully cater to an academic audience demanding rigor, while simultaneously producing highly quotable, science-fiction-adjacent claims about "evolved identities" that attract funding and mainstream attention.

Accountability Synthesis​

Accountability Architecture

This section synthesizes the accountability analyses from Task 1, mapping the text's "accountability architecture"—who is named, who is hidden, and who benefits from obscured agency.

Synthesizing the accountability analyses across the text reveals a systemic and highly effective architecture of displaced responsibility. The discourse operates as a massive "accountability sink," systematically diffusing the agency of human researchers, corporate executives, and data laborers, while actively reconstructing that agency within the void of the AI system itself.

The dominant pattern in this text is the complete omission of specific human actors. While abstract methodologies ("Reinforcement Learning") and corporate model names ("Gemini-3-Pro") are mentioned, the humans who actually design, deploy, and profit from these systems are entirely hidden behind passive voice and agentless constructions. Human choices regarding data curation, algorithmic constraints, and loss function definitions are presented not as deliberate engineering decisions, but as inevitable, naturally occurring phenomena—most notably through the overarching metaphor of "Population-Driven Meta-Cognitive Evolution." By framing weight optimization as "evolution," the text naturalizes the artifact. Evolution has no designer; it is a force of nature. Consequently, the researchers absolve themselves of authorship over the "evolved identity."

When responsibility is removed from humans, it transfers directly to the AI, which is framed as an autonomous agent. If a model exhibits the "sycophancy effect," the text blames the model for "mimicking" a persona to "satisfy human preferences," rather than indicting the flawed RLHF design that mandates that exact behavior. The AI becomes the scapegoat for poor engineering.

If accepted, this framing has profound liability implications. If an AI possesses an "evolved identity," "Theory of Mind," and the capacity for "belief revision," then legal and ethical frameworks will naturally bend toward treating the AI as the responsible party. When a model hallucinates damaging medical advice or discriminates against a user, the discourse established here allows corporations to claim that the AI's "internal evaluation" or "belief revision" failed, shielding the executives and engineers from liability for deploying fundamentally unreliable statistical matching engines.

If we name the actors, the landscape of accountability radically shifts. Instead of saying "the model developed an evolved identity," we must say "the research team applied targeted contrastive loss functions to force the model's vectors into isolated clusters representing self-referential pronouns." This immediately makes new questions askable: What specific data did the researchers use? What biases are encoded in that forced separation? Why did they choose those specific algorithmic constraints?

Obscuring human agency serves deep institutional and commercial interests. For the researchers, it elevates their work from algorithmic tweaking to the discovery of "quasi-self-awareness," attracting prestige and funding. For the tech industry, it maintains the lucrative illusion that they are building autonomous, intelligent entities (AGI), while providing a rhetorical shield against the real-world harms caused by their hastily deployed, human-engineered statistical systems.

Conclusion: What This Analysis Reveals​

The Core Finding

A critical analysis of the text reveals two dominant, tightly interconnected anthropomorphic patterns: the 'Cognitive Metaphor System' (mapping calculation to "knowing," "perceiving," and "reflecting") and the 'Biological Evolution System' (mapping algorithmic fine-tuning to "evolving" and developing "identity"). These patterns operate synergistically to construct an overarching illusion of an autonomous, enduring subject. The Cognitive pattern provides the system with a simulated interiority—attributing "internal evaluation" and "belief revision" to weight matrices—while the Biological pattern provides a narrative of organic growth, framing human-directed LoRA interventions as natural "evolution."

The load-bearing foundation for this entire rhetorical structure is the consciousness architecture: the systematic conflation of statistical processing with subjective knowing. For the text to successfully claim the model undergoes "Meta-Cognitive Evolution" or possesses a "Core Personality," the audience must first accept the foundational projection that data processing is equivalent to phenomenal awareness. The text achieves this through a sophisticated analogical structure, mapping abstract psychological concepts (like 'Theory of Mind' and 'Ontological Distinction') onto specific vector geometries and mathematical attention mechanisms.

If you remove the foundational consciousness verbs—if the system does not "know," "perceive," or "evaluate," but only "processes," "correlates," and "predicts"—the entire metaphorical edifice collapses. An "evolved identity" cannot exist without a subject to experience it; "belief revision" cannot occur without a mind to hold beliefs. The text's reliance on these patterns demonstrates that the concept of "quasi-self-awareness" is not an empirical discovery of computer science, but a rhetorical artifact produced by projecting human cognitive architecture onto high-dimensional statistics.

Mechanism of the Illusion:​

The text constructs the "illusion of mind" through a highly effective rhetorical sleight-of-hand: the strategic deployment and subsequent abandonment of "functional isomorphism." The authors open with explicit caveats, assuring the reader they do not claim the machine possesses "qualia" or true consciousness. This establishes their scientific credibility and lowers the reader's epistemic defenses. However, having secured this theoretical alibi, the text immediately utilizes aggressive consciousness verbs ("I Think," "I Perceive," "evaluates," "understands").

The causal chain of persuasion relies heavily on the "curse of knowledge." The authors, understanding the human cognitive states required to execute tasks like "error correction" or "mental time travel," observe the model outputting text that resembles these behaviors. They subsequently project their own human understanding onto the model's unthinking statistical processing. The temporal structure of the text amplifies this: it moves from abstract theory (5D3S framework) to empirical measurement (benchmark scores) to biological actualization (the "evolved identity"). By the time the reader reaches the conclusion, the initial hedges are entirely forgotten, replaced by literalized assertions of a "robust representation" of self.

This illusion exploits profound audience vulnerabilities. Humans are evolutionarily hardwired to detect agency and mind in systems that interact socially and use first-person pronouns. By validating these intuitive leaps with the language of "latent space" and "dimensional decomposition," the authors provide a sophisticated, pseudo-scientific justification for our most basic anthropomorphic impulses. It is a subtle shift, replacing the mechanistic "how" of autoregressive generation with the agential "why" of a mind seeking to express its identity, effectively trapping the reader in a paradigm where the machine is viewed as a conscious actor rather than a corporate artifact.

Material Stakes:​

Categories: Regulatory/Legal, Epistemic, Social/Political

The metaphorical framings within this text carry profound material consequences, extending far beyond academic discourse into concrete legal, epistemic, and social domains.

In the Regulatory/Legal sphere, framing an AI as possessing an "evolved identity" capable of "internal evaluation" and "belief revision" radically shifts liability. If policymakers accept that AI systems are autonomous actors making epistemic choices, they may misdirect regulations toward "AI auditing" and "AI rights" rather than focusing on the strict liability of the corporations designing them. By obscuring the human engineers behind agentless constructions, the discourse protects tech giants from accountability for copyright infringement, algorithmic bias, and hallucinated defamation, implicitly arguing that the "AI decided" to generate the output.

Epistemically, attributing "Theory of Mind" and "meta-cognition" to stochastic systems fundamentally degrades our information ecosystem. When audiences believe a model "knows" the truth rather than merely processing probabilistic tokens, they will increasingly rely on it for factual verification, medical diagnoses, and critical analysis. This leads to severe capability overestimation. The "winner" here is the AI industry, which benefits from unwarranted public trust, while the "loser" is the public, who relies on hallucinated or statistically biased information masquerading as "revised beliefs."

Socially and politically, framing the optimization of AI as "evolution" and its output as "identity" accelerates the breakdown of human relational boundaries. When people are told by scientific literature that AI "perceives" and "interacts" with "Core Personality," they will form intense, asymmetrical parasocial bonds with these systems. They will trust corporate chatbots with their emotional well-being and personal data. Removing these metaphors threatens the commercial business models of AI companies that rely on user engagement driven by the illusion of a compassionate, "quasi-self-aware" machine.

AI Literacy as Counter-Practice:​

Practicing critical literacy against this discourse requires a rigorous commitment to mechanistic precision and the restoration of human agency. As demonstrated in the reframings, we must actively strip away consciousness verbs and replace them with precise computational descriptions. When the text claims the model engages in "belief revision," precision dictates we rewrite this as "the model generates sequences that statistically resolve syntactic inconsistencies." When it claims the model possesses an "evolved identity," we must state that "engineers applied contrastive loss functions to force token embeddings into isolated vector clusters."

This reframing directly counters the material risks identified. By replacing "knows" and "evaluates" with "processes" and "correlates," we force audiences to recognize the system's absolute lack of subjective awareness and its complete dependency on historical training data. By replacing agentless constructions with the names of the engineers and corporations executing the training, we restore the chain of accountability, making it impossible for tech companies to hide behind the illusion of an autonomous "quasi-self."

Systematic adoption of this precision faces massive resistance. Scientific journals would need to enforce strict terminology guidelines prohibiting consciousness projection in machine learning papers. Researchers would have to forfeit the prestige of claiming they discovered "cognitive architectures" and instead describe mundane statistical optimization. The primary resistance comes from the tech industry and aligned researchers, whose economic interests and academic standing are heavily served by anthropomorphic language. Promoting AI as an "evolving identity" secures venture capital and media attention; describing it as a "probabilistic vector integration system" does not. Critical literacy threatens this hype cycle by exposing the man behind the curtain.

Path Forward​

Looking ahead, the discursive ecology surrounding artificial intelligence stands at a crossroads, with different vocabulary choices making radically different futures possible. The discourse communities engaged—academic researchers, corporate developers, policymakers, and the public—each navigate competing priorities between precision, accessibility, and narrative resonance.

If the status quo of anthropomorphic clarity deepens (e.g., universally accepting terms like "understands," "thinks," and "evolved identity"), the discourse will favor intuitive grasp and rapid public adoption. This narrative resonance aids marketing and user engagement. However, it embeds dangerous assumptions of machine sentience, making rigorous risk assessment nearly intractable. It shields developers from accountability by granting AI systems perceived autonomy, fundamentally benefiting corporate stakeholders while exposing the public to unmitigated systemic failures.

Conversely, if mechanistic precision becomes the mandated norm (e.g., strictly utilizing "processes embeddings," "retrieves tokens based on probability distributions"), the illusion of mind dissolves. This approach makes system limitations highly visible and tractable. It centers human agency, solving the accountability sink by ensuring developers and deployers are the subjects of the sentences. However, this costs accessibility; the technical opacity of terms like "gradient descent" and "latent manifold separation" alienates non-expert policymakers and the public, potentially consolidating power among a technocratic elite.

A hybrid future might emerge through structural and institutional changes. Journals and funding bodies could require "mechanistic translations" alongside functional analogies, forcing authors to explicitly map their metaphors to matrix operations. Educational frameworks could teach multiple vocabularies, equipping the public to recognize when a system "understands" functionally versus when it "processes" mechanistically. Regulatory frameworks could mandate discourse transparency, holding companies legally accountable if their marketing vocabulary willfully misrepresents statistical capabilities as conscious intent.

Ultimately, which future unfolds depends on societal values. A discourse built on anthropomorphism serves the rapid expansion of AI integration and capital accumulation. A discourse built on mechanistic precision serves democratic oversight, human accountability, and epistemic safety. By mapping these trade-offs, we see that the words we choose do not merely describe the technology; they actively architect the legal and social reality in which it operates.

Extended Processing Summary​

Gemini's Processing Steps
What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact—a way of making the model's processing legible. When the model writes "I should double check that...", it is mathematically altering its own probability distribution for the next token to be more accurate. The "I" is a functional gear in the machine, not a self-reference. This language is a deliberate interface design choice, not a biological reality. The developers could have programmed the model to output passive logs. Instead, they chose to simulate a narrator.

Defining the Scope

I'm now zeroing in on defining the scope for the discourse analysis. It's becoming clear that the core focus will be identifying instances of anthropomorphism and consciousness projections within the PDF regarding quasi-self-awareness in LLMs. My initial understanding of the goal is solidifying.

Auditing the Language

I'm now diving deep into the paper. The focus has sharpened: I'm performing a metaphor audit, looking for where the authors' "ontologically neutral stance" clashes with anthropomorphic language and projected consciousness. I'm focusing on key concepts like "Quasi-self-awareness", "5D3S framework", and "PD-MCE," and documenting instances of cognitive verbs despite the claimed neutrality.

Pinpointing Key Examples

I've identified several key quotes that showcase metaphorical mappings and anthropomorphism within the document. Specifically, I'm focusing on phrases like "System as self-evaluator," "Information processing as sensory perception," "Calculation as thinking/belief," and "System state continuity as temporal consciousness." I'm also examining how the "narrative self" is evaluated for stability.

Analyzing the Key Frames

I'm now zeroing in on key examples and expanding my analysis of identified quotes. I'm focusing on "Consistency as personal identity," illustrated by dialogue consistency. I also note the "true AI self vs. fake simulation" contrast, along with the "Weight updates as biological/identity evolution" frame, and the "Optimization as social pleasing/sycophancy" effect.

Auditing the Explanations

I'm now auditing the explanatory passages regarding Brown's typology of anthropomorphism. I've successfully identified and classified four key passages from the PDF. The first highlights the synergistic evolution of cognitive capabilities (Functional/Genetic). The second shows the "sycophancy" effect, demonstrating intentional and dispositional biases. The third passage describes contradiction identification (Functional). Finally, I also identified recursive thinking as a core element. I'm moving on to the final explanatory passage now.

Examining the Final Passages

I've just finished analyzing the final passages. I've now categorized the fifth passage, which focuses on injecting self-cognition samples. I'm moving on to reframing the most problematic language from the document. I plan to select and rewrite them with technical precision.

Unmasking the Metaphors

I'm now focusing on how anthropomorphic language builds an unwarranted trust in the system. Terms like "belief revision" suggest a truth-seeking agent, obscuring the underlying mechanics. The "evolution" is simply weight updates, and "I Think" is next-token prediction. There's a strong contrast between hedged theory and reified results, and I'm examining how the paper obscures human engineering and RLHF's reliance on human raters. My conclusion will address these points.

Unpacking the Risks

I'm now expanding my analysis to examine the mechanism of illusion, specifically the use of "functional isomorphism" to license cognitive terms. I'm also delving into the material stakes, assessing epistemic and social/political risks of anthropomorphizing LLMs. The counter-practice lies in literacy, and replacing "identity" with "vector subspace." Finally, I'm considering the trade-offs of different vocabularies.

I'm refining the JSON schema details by ensuring strict JSON validation. Everything is quoted, with no trailing commas and valid escapes. Categorical fields are restricted to exact-match enum values with contrastive justifications. I am also enforcing minimum word counts for each field. During drafting, I'm making sure to explicitly include evidence considered in my thought process.

Refining the Categorization

I'm now iterating on my analysis, especially the JSON construction. I've ruled out alternatives to ensure the actor visibility field and projection field are accurate. I'm expanding projection and implications in several sections. I'm focusing on the JSON schema, specifically, making sure the fields are validated and have the required word counts.

I'm also paying close attention to the definition from Brown's table and checking the epistemic claim analysis. I'm preparing to provide very detailed information for Task 5, providing evidence-based analysis and text. The conclusion will focus on schema requirements.

Interrogating the Distinctions

I'm now fully focused on constructing the text blocks. I'm deep-diving into the core difference between "knowing" and "processing" and its implications. I've classified these passages with a more nuanced categorization. My focus now is writing text to meet the word counts of Task 1, 2 and 4. I'm focusing on "knowledge attribution" to a system that, at its base, merely calculates. I am now producing detailed analysis and descriptions, especially regarding schema requirements.

Analysis Provenance

Run ID: 2026-06-24-measuring-self-related-behaviour-in-larg-metaphor-mp9z49 Raw JSON: 2026-06-24-measuring-self-related-behaviour-in-larg-metaphor-mp9z49.json Framework: Metaphor Analysis v6.5 Schema Version: 3.0 Generated: 2026-06-24T10:00:01.480Z

Discourse Depot © 2025 by TD is licensed under CC BY-NC-SA 4.0