Skip to main content

🆕+📊 Language Statistics and False Belief Reasoning: Evidence from 41 Open-Weight LMs

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping), the philosophy of social science (Robert Brown's typology of explanation), and accountability analysis.

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.

Task 1: Metaphor and Anthropomorphism Audit​

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

V3 Enhancement: Each metaphor now includes an accountability analysis.

1. Cognitive Action as Statistical Correlation​

Quote: "Research on mental state reasoning in language models (LMs) has the potential to inform theories of human social cognition..."

  • Frame: Model as cognitive reasoner
  • Projection: This metaphor maps the deeply human, conscious capacity for 'mental state reasoning' onto a computational system. By using the word 'reasoning,' the text projects justified true belief, conscious deliberation, and subjective awareness onto what is mechanistically a statistical pattern-matching process. It attributes the act of 'knowing'—the conscious comprehension of another being's internal mental landscape—to a system that merely 'processes' token probabilities and word co-occurrences. This suggests the AI actively understands human psychology and possesses a Theory of Mind, fundamentally blurring the absolute ontological distinction between a conscious human organism with empathetic awareness and a mathematical artifact performing matrix multiplications.
  • Acknowledgment: Direct (Unacknowledged) (The phrase 'Research on mental state reasoning in language models' is stated as a literal, objective fact outlining the paper's domain without any hedging, scare quotes, or qualifications.)
  • Implications: Framing an AI as capable of 'mental state reasoning' drastically inflates its perceived sophistication and creates severe risks of unwarranted trust. If users believe a system can genuinely 'reason' about their 'mental states,' they may inappropriately rely on it for sensitive tasks like psychological counseling, interpersonal conflict resolution, or legal mediation. It obscures the reality that the system cannot comprehend human intent or emotion, leading to dangerous policy implications where automated systems might be deployed in high-stakes social environments under the false premise that they possess empathy or social intelligence. Liability becomes deeply ambiguous when failures are attributed to the AI's flawed 'reasoning' rather than to human design flaws.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The text entirely obscures the human actors—the researchers, dataset curators, and model engineers at companies like Meta, Google, and AllenAI—who designed the architecture, selected the training data, and established the objective functions. By framing the AI as the autonomous entity performing 'mental state reasoning,' the agency of the developers who embedded these statistical correlations into the system is completely hidden. If the actors were named, we would recognize that human engineers designed a system that mimics human text patterns relating to psychology. The agentless construction serves the interests of the AI industry by making the system appear intellectually advanced, distracting from corporate design choices.
Show more...

2. Machine as Biological Entity​

Quote: "...evaluating the cognitive capacities of LMs or using LMs as 'model organisms' to test (or generate) hypotheses about human cognition."

  • Frame: Model as biological organism
  • Projection: This metaphor maps the properties of living, biological entities onto a static software artifact. The projection attributes 'cognitive capacities' to the system, suggesting the AI possesses intrinsic, organic thought processes similar to a living creature used in laboratory experiments. By describing the AI as a 'model organism' possessing 'capacities,' the text projects the biological reality of learning, knowing, and experiencing onto a system that only executes programmed mathematical operations. It conflates the mechanical processing of weights and biases with the organic, conscious knowing of a living subject, inviting the audience to view the algorithm as a form of emergent synthetic life.
  • Acknowledgment: Explicitly Acknowledged (The text uses scare quotes around 'model organisms,' explicitly acknowledging the metaphorical transfer from biology to computer science in this specific instance.)
  • Implications: Designating software as a 'model organism' fundamentally distorts the public and regulatory understanding of AI. It suggests that AI behavior is an organic, naturally occurring phenomenon that must be discovered or studied like biology, rather than an engineered product that was deliberately constructed by humans. This inflates perceived capability and naturalizes the technology, making its flaws seem like natural biological variations rather than specific engineering errors. It shields developers from accountability by implying that the system has a life of its own, out of the direct control of its creators, thereby complicating legal liability for harmful outputs.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The construction hides the human creators who engineered these specific systems. AI systems do not naturally exist as 'organisms' in the wild; they are built by corporate teams pursuing specific commercial goals. Naming the actors would mean stating that researchers use commercial software products built by major tech companies to model human behavior. The 'organism' metaphor actively serves the AI industry by naturalizing their products, making them seem like independent scientific phenomena rather than proprietary software subject to human flaws, bias, and corporate governance.

3. Correlation as Empathetic Awareness​

Quote: "LMs exhibit some sensitivity to canonical belief-state manipulations..."

  • Frame: Model as perceptive entity
  • Projection: The term 'sensitivity' projects conscious perception, emotional awareness, and cognitive receptivity onto the AI. It maps the human ability to 'know' and 'feel' nuances in another person's belief state onto the model's mechanical capacity to output different tokens when input prompts are altered. This projects a deeply conscious state—awareness of another's subjectivity—onto the rigid mechanics of gradient descent and attention calculation. It implies the model actively understands and reacts to the meaning of the text, rather than passively correlating string inputs with statistically probable string outputs based on its training distribution.
  • Acknowledgment: Hedged/Qualified (The phrase is slightly qualified with the word 'some', suggesting a limitation or boundary on the extent of the model's perceptive capability.)
  • Implications: Attributing 'sensitivity' to belief states encourages users to anthropomorphize the system as an empathetic or emotionally intelligent agent. This false perception of social awareness can lead vulnerable users to form deep, relation-based trust with the machine, sharing private data or relying on it for emotional support. It dangerously overestimates the model's capabilities, masking the fact that its 'sensitivity' is merely a change in statistical probability, not a genuine comprehension of truth or human context. This creates liability risks when the model inevitably fails to handle complex human emotional situations safely.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The text obscures the human experimental designers who construct the 'canonical belief-state manipulations' (the prompts) and the developers who gathered the data that allows the model to respond differentially. The model does not actively 'exhibit sensitivity'; rather, it mathematically reflects the semantic patterns embedded in its training data by human engineers. If human agency were restored, the text would clarify that the researchers' manipulations of input strings reliably trigger different statistical outputs from the model. Displacing this agency onto the AI creates an illusion of independent social intelligence.

4. Prediction as Active Judgment​

Quote: "LMs and humans more likely to attribute false beliefs in the presence of non-factive verbs like 'thinks'..."

  • Frame: Model as active adjudicator
  • Projection: The verb 'attribute' projects conscious judgment and the possession of a conceptual framework onto the AI system. To 'attribute a false belief' requires an entity to possess a conscious understanding of truth, an awareness that another entity holds a contrary belief, and the active cognitive intention to assign that state to them. By equating LMs and humans in their ability to 'attribute,' the text maps human justified knowing onto machine processing. It treats the generation of a statistically probable text string containing an incorrect location as a conscious act of psychological attribution, fundamentally confusing computation with cognition.
  • Acknowledgment: Direct (Unacknowledged) (The text directly equates LMs and humans in their propensity to 'attribute false beliefs' without any hedging or acknowledgment of the fundamental mechanical differences.)
  • Implications: By claiming that models 'attribute false beliefs,' the discourse grants AI systems the status of active evaluators of human truth and falsehood. This inflates the model's perceived authority, suggesting it can reliably judge the epistemic states of users or subjects. If policymakers or legal professionals believe an AI can accurately 'attribute beliefs,' they might deploy such systems to detect deception, assess intent in criminal cases, or evaluate psychological fitness. This poses extreme risks, as the system is merely predicting text based on lexical co-occurrence (e.g., 'thinks' correlates with incorrect statements in the training data), lacking any actual evaluative capacity.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The text equates 'LMs' directly with 'humans' as actors, erasing the actual humans who built the LMs. The models do not 'attribute' anything; rather, the engineers who compiled the massive training datasets captured human linguistic patterns where non-factive verbs co-occur with false statements. The researchers then prompt the model, triggering this statistical association. Replacing the LM as the actor with the human developers would reveal that the models simply reproduce human biases encoded by their creators. The agentless framing absolves creators of responsibility for the biases their systems perpetuate.

5. Optimization as Organic Growth​

Quote: "...what aspects of human cognition can emerge in a learner trained purely on the distributional statistics of language."

  • Frame: Model as developing student
  • Projection: Calling the AI a 'learner' projects human educational, intellectual, and developmental qualities onto a mathematical optimization process. It maps the conscious, subjective experience of acquiring knowledge onto the mechanical procedure of adjusting network weights via backpropagation. It suggests the system is an active agent seeking understanding ('learning' and 'knowing'), rather than a passive repository of statistical correlations ('processing'). The word 'emerge' further projects a sense of organic, spontaneous biological or cognitive development, masking the highly controlled, mathematically rigid process of model training engineered by humans.
  • Acknowledgment: Direct (Unacknowledged) (The system is directly labeled as a 'learner' in a declarative sentence about what cognitive aspects can 'emerge', presented without qualification.)
  • Implications: The 'learner' metaphor invokes powerful human frameworks of education, innocence, and organic growth, which systematically lowers the audience's threat perception. If the AI is just a 'learner,' its errors are viewed sympathetically as 'mistakes' along an educational journey rather than as critical failures of a commercial product. This anthropomorphism severely hampers critical technological evaluation, encouraging the public to extend the patience and relation-based trust they would give to a human student to a multi-billion-dollar corporate algorithm. It obscures the rigid determinism of the system's architecture.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: By framing the system as a 'learner' in which cognition might 'emerge,' the text totally eclipses the massive corporate teams that actually 'train' the model. Models do not learn spontaneously; human data engineers curate petabytes of text, reinforcement learning teams write specific reward functions, and executives dictate optimization goals. If the text named the actors, it would state: 'what capabilities tech companies can engineer into software by applying optimization algorithms to human text.' The 'learner' framing diffuses corporate accountability by presenting the AI's capabilities and flaws as emergent, natural phenomena rather than engineered choices.

6. Mathematical Adjustment as Skill Development​

Quote: "LMs trained on the distributional statistics of language can develop sensitivity to implied belief states..."

  • Frame: Model as maturing subject
  • Projection: The phrase 'develop sensitivity' projects a human narrative of emotional and psychological maturation onto the AI. It maps the conscious human experience of gradually coming to know and understand complex social dynamics onto the static, mechanical reality of a pre-trained neural network processing inputs. 'Developing sensitivity' implies a conscious awakening to nuance and a capacity for justified belief, whereas the system merely processes text tokens through fixed mathematical weights. It attributes the deeply human quality of social knowing to an artifact that is simply executing a complex but entirely mechanistic classification task.
  • Acknowledgment: Direct (Unacknowledged) (The assertion that LMs 'can develop sensitivity' is stated as a direct, unhedged scientific conclusion regarding the models' capabilities.)
  • Implications: This framing radically inflates the perceived emotional and psychological depth of the software. By suggesting the AI can 'develop sensitivity,' it invites users to treat the system as a socially aware entity capable of nuanced interpersonal engagement. This poses massive risks for unwarranted trust, especially in mental health or customer service applications, where users may assume the AI truly grasps the subtleties of their emotional state. It shifts the public understanding of AI from a predictable, mechanical tool to an unpredictable, emotionally maturing agent, complicating how we assess its reliability and safety.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The text attributes the action of 'developing' to the LMs themselves, obscuring the engineers who updated the model weights or the researchers who crafted the specific prompts that elicited the behavior. The AI does not actively develop anything; its parameters were fixed during training by human engineers, and its outputs are mechanically generated. Identifying the human actors would reveal that corporate developers tuned the models to produce outputs that mimic human social awareness. Obscuring this fact grants the software a false autonomy that deflects scrutiny away from its corporate creators.

7. Mechanistic Failure as Cognitive Fragility​

Quote: "...although LMs are surprisingly capable on mental state reasoning tasks, their performance remains relatively brittle..."

  • Frame: Model as fragile intellect
  • Projection: The text pairs the highly cognitive term 'capable on mental state reasoning' with the term 'brittle.' While 'brittle' is often used mechanistically in software engineering, here it is mapped onto a cognitive capacity, projecting the image of a system that genuinely 'knows' how to reason but gets easily confused or overwhelmed. This projects the human experience of cognitive vulnerability or mental exhaustion onto a system that is simply failing to find statistical correlations in its training data due to novel prompt phrasing. It maintains the illusion that the system is 'thinking,' even when it fails.
  • Acknowledgment: Hedged/Qualified (The claim is heavily qualified with the adverbs 'surprisingly' and 'relatively', indicating a tension or limitation in the model's capabilities.)
  • Implications: Describing a system's failures as 'brittle mental state reasoning' rather than 'statistical misclassification' preserves the illusion of the AI's general intelligence even in the face of failure. It encourages users and policymakers to view the AI as fundamentally intelligent but occasionally prone to 'mistakes,' much like a human. This prevents audiences from understanding that the system never actually understood the task in the first place; it only succeeded previously because the prompt matched its training data. This misunderstanding leads to dangerous overestimations of the system's reliability in novel, real-world situations.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The text frames the AI as the subject that possesses both 'capability' and 'brittleness,' hiding the human designers whose limited training datasets and specific architectural choices caused the system to fail on altered prompts. The model's failure is not an internal cognitive fragility, but a direct result of the developers' failure to provide sufficiently diverse training data. If human actors were named, the sentence would read: 'systems built by AI companies fail when researchers alter the prompts because the engineers' training data lacked sufficient variation.' Agentless language protects the developers from criticism of their dataset curation.

8. Mathematical Output as Deceptive Imputation​

Quote: "...imputing an incorrect belief to an agent when a non-factive verb is used..."

  • Frame: Model as interpreting adjudicator
  • Projection: The verb 'impute' projects a high level of conscious intent, subjective judgment, and active knowing onto the language model. To 'impute' a belief requires the conscious recognition of another distinct entity (an agent), an understanding of what a belief is, and the cognitive action of assigning that state to them. By using this term, the text maps complex social cognition onto a system that merely processes strings of text. It suggests the model actively evaluates reality and knowingly assigns an incorrect state, whereas the system is simply generating the token sequence with the highest probability given the input.
  • Acknowledgment: Direct (Unacknowledged) (The phrase 'imputing an incorrect belief to an agent' is used directly to describe the mechanism of the model's behavior without any metaphorical acknowledgment.)
  • Implications: This deep consciousness projection endows the machine with the illusion of sophisticated social reasoning and active interpretive power. If the public and regulators believe that AI systems can actively 'impute beliefs,' they will severely overestimate the system's ability to navigate complex social, legal, or ethical scenarios. This creates a dangerous liability gap: if a system makes a harmful classification, the anthropomorphic framing suggests the AI made a 'bad judgment' or 'imputed incorrectly,' rather than forcing accountability onto the corporate engineers whose biased or incomplete training data guaranteed that specific statistical output.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The text presents the action of 'imputing' as something the model does autonomously, completely obscuring the role of the training data and its human curators. The system does not impute beliefs; rather, human developers trained the system on human-generated text where non-factive verbs statistically correlate with false statements. The researchers then prompted the system to reveal this correlation. If human agency were restored, we would recognize that human language patterns, captured and embedded by corporate engineers, dictate the output. The agentless framing masks the structural reality of the model's dependency on human data.

Task 2: Source-Target Mapping​

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: Conscious human reasoner → Statistical token prediction based on False Belief task prompts​

Quote: "Research on mental state reasoning in language models (LMs)..."

  • Source Domain: Conscious human reasoner
  • Target Domain: Statistical token prediction based on False Belief task prompts
  • Mapping: The relational structure of a human consciously evaluating a social situation—involving empathy, an internal model of another's mind, and logical deliberation—is mapped directly onto the AI's processing of text prompts. This mapping invites the assumption that the language model possesses an internal epistemology and the capacity for justified belief. It projects the conscious state of 'knowing' a psychological concept onto the purely mechanistic act of processing vector embeddings and outputting the most statistically probable string of words.
  • What Is Concealed: This mapping completely conceals the mechanical reality of matrix multiplication, attention mechanisms, and gradient descent. It hides the fact that the system possesses no internal world model, no subjective experience, and no actual comprehension of what a 'mental state' is. Transparency is heavily obstructed here: the text makes claims about the model's 'reasoning' while obscuring the proprietary training data and specific corporate optimization choices that actually generated the statistical correlations the model is regurgitating.
Show more...

Mapping 2: Biological living organism → Engineered software and mathematical weights​

Quote: "...evaluating the cognitive capacities of LMs or using LMs as 'model organisms'..."

  • Source Domain: Biological living organism
  • Target Domain: Engineered software and mathematical weights
  • Mapping: The structure of biological science—where scientists study naturally occurring, living entities with inherent, organic traits—is mapped onto computer science. The mapping assumes that AI models have internal 'cognitive capacities' that grow and exist independently of their creators, just like a lab mouse. It projects the organic, conscious reality of living, breathing, and knowing onto static, human-engineered code, suggesting the AI's behavior is a natural phenomenon rather than a product of specific mathematical algorithms.
  • What Is Concealed: This biological metaphor deeply conceals the engineered, artificial, and commercial nature of language models. It hides the human labor, corporate decision-making, and immense environmental resources required to build these systems. By treating the model as an 'organism,' it rhetorically exploits the opacity of complex software, masking the fact that its behavior is dictated by deterministic code and curated datasets created by specific companies like Meta or Google, not by natural biological evolution.

Mapping 3: Empathetic, perceptive human observer → Differential statistical outputs based on varied input strings​

Quote: "LMs exhibit some sensitivity to canonical belief-state manipulations..."

  • Source Domain: Empathetic, perceptive human observer
  • Target Domain: Differential statistical outputs based on varied input strings
  • Mapping: The source domain of a human being emotionally or cognitively 'sensitive' to the subtle mental states of others is projected onto the target domain of a neural network generating different outputs when input tokens are changed. This invites the assumption that the machine has a conscious, perceptive awareness of the meaning behind the text. It maps the act of conscious 'knowing' and social empathy onto the mechanistic process of classifying prompt variations.
  • What Is Concealed: The mapping conceals the rigid, mathematical nature of the model's operations. It hides the fact that the system does not 'feel' or 'perceive' anything; it merely calculates probabilities based on the proximity of vectors in high-dimensional space. It obscures the direct dependency on the human researchers who engineered the 'manipulations' and the corporate engineers who provided the training data, falsely presenting a statistical correlation as an internal, empathetic trait of the machine.

Mapping 4: Conscious adjudicator of truth → Probability distributions reflecting lexical co-occurrences​

Quote: "LMs and humans more likely to attribute false beliefs in the presence of non-factive verbs..."

  • Source Domain: Conscious adjudicator of truth
  • Target Domain: Probability distributions reflecting lexical co-occurrences
  • Mapping: This maps the deeply human, conscious act of judging truth claims and 'attributing' internal states to others onto a system's statistical tendency to output certain words together. It projects the conscious requirement of holding a justified belief and understanding the concept of falsehood onto a machine. By placing LMs and humans in the same functional category, the mapping assumes that the machine's text generation is driven by the same epistemological and cognitive processes that drive human psychological evaluation.
  • What Is Concealed: This mapping hides the utter lack of ground truth or semantic understanding within the AI system. It conceals the mechanistic reality that the model only outputs incorrect locations because words like 'thinks' statistically co-occur with false statements in the massive human datasets it ingested. It obscures the role of the humans who generated that original text and the engineers who scraped it, attributing human-like active judgment to a system that only executes passive pattern matching.

Mapping 5: Human student in an educational environment → Iterative weight updates in a neural network​

Quote: "...what aspects of human cognition can emerge in a learner trained purely on the distributional statistics..."

  • Source Domain: Human student in an educational environment
  • Target Domain: Iterative weight updates in a neural network
  • Mapping: The relational structure of a human student actively acquiring knowledge, growing intellectually, and developing cognition is mapped onto the algorithmic process of updating parameters to minimize loss. The mapping invites the assumption that the system possesses a conscious drive to 'know' and understand its environment. It projects the subjective experience of learning and organic cognitive 'emergence' onto the highly controlled, mathematically rigorous procedure of backpropagation.
  • What Is Concealed: This educational metaphor conceals the intense corporate engineering, human labor, and computational force required to 'train' these models. It hides the RLHF (Reinforcement Learning from Human Feedback) workers, the data annotators, and the algorithm designers whose explicit choices determine the system's output. By framing the system as a spontaneous 'learner,' the text obscures the proprietary opacity of the training data and exploits the metaphor to make the technology seem natural and benign rather than an engineered corporate product.

Mapping 6: Maturing human psychology → Fixed mathematical parameters classifying text​

Quote: "LMs trained on the distributional statistics of language can develop sensitivity to implied belief states..."

  • Source Domain: Maturing human psychology
  • Target Domain: Fixed mathematical parameters classifying text
  • Mapping: The human process of psychological maturation—gradually coming to understand and 'know' complex social and emotional nuances—is projected onto the static, trained weights of a language model. This mapping assumes that the AI possesses an internal subjectivity capable of growth and deep comprehension. It projects conscious awareness and empathetic knowing onto an artifact that merely processes data according to mathematical rules, suggesting the system is actively awakening to human social dynamics.
  • What Is Concealed: The mapping conceals the fact that the model's parameters are fixed after training; it does not 'develop' anything during inference. It hides the mechanical reality that the model is simply matching patterns based on the statistical distribution of its training data. This language obscures the agency of the corporate developers who tuned the model to generate responses mimicking social awareness, falsely presenting their engineering success as the AI's personal psychological development.

Mapping 7: Fragile human intellect → Statistical failure due to out-of-distribution inputs​

Quote: "...although LMs are surprisingly capable on mental state reasoning tasks, their performance remains relatively brittle..."

  • Source Domain: Fragile human intellect
  • Target Domain: Statistical failure due to out-of-distribution inputs
  • Mapping: The source domain of a human mind that is intelligent but susceptible to confusion, exhaustion, or cognitive fragility is mapped onto a computer program's failure to process novel prompts accurately. This projection assumes that the model possesses genuine 'reasoning' capabilities that simply break down under pressure. It maps the conscious experience of mental failure onto the mechanistic reality of a system failing to find statistical correlations because the input data deviates from its training distribution.
  • What Is Concealed: This mapping conceals the fundamental absence of intelligence in the system. It hides the mechanical reality that the AI never 'reasoned' correctly in the first place; its prior successes were merely statistical reflections of its training data. By calling it 'brittle reasoning,' the text obscures the developers' failure to provide robust, diverse datasets, masking a human engineering flaw as an internal cognitive quirk of the machine.

Mapping 8: Active human interpreter and judge → Generation of high-probability tokens​

Quote: "...imputing an incorrect belief to an agent when a non-factive verb is used..."

  • Source Domain: Active human interpreter and judge
  • Target Domain: Generation of high-probability tokens
  • Mapping: The structure of a human consciously interpreting a situation and actively assigning a specific, justified belief to another person is mapped onto the algorithm's generation of text. This projection assumes the AI has an epistemological framework—that it 'knows' what a belief is, understands the concept of an 'agent,' and actively chooses to assign an incorrect status. It maps deep, conscious social cognition onto the mechanistic process of retrieving text tokens that correlate with input strings.
  • What Is Concealed: This mapping totally conceals the mathematical nature of the model's text generation. It hides the fact that the system possesses no agency, no understanding of truth or falsity, and no concept of other 'agents.' It obscures the reality that the system is simply reproducing the statistical patterns of human language—specifically, the linguistic correlation between non-factive verbs and false statements—embedded in its training data by human engineers, presenting computation as a conscious, interpretive act.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")​

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1​

Quote: "LMs trained on the distributional statistics of language can develop sensitivity to implied belief states..."

  • Explanation Types:

    • Genetic: Traces origin through dated sequence of events or stages
    • Empirical Generalization: Subsumes events under timeless statistical regularities

  • Analysis (Why vs. How Slippage): This explanation exhibits a profound slippage between mechanistic and agential framing. The first half ('trained on the distributional statistics') provides a highly mechanistic, Empirical Generalization explaining the 'how'—the model relies on mathematical probabilities derived from data. However, the second half ('develop sensitivity to implied belief states') shifts abruptly to an agential, Genetic explanation of 'why' it behaves this way, framing the outcome as an organic, cognitive maturation. This hybrid choice emphasizes the model's perceived sophistication by grounding it in technical reality but elevating it through developmental psychology terminology. It actively obscures the fact that 'sensitivity' is just a metaphor for generating statistically probable text strings, masking the human engineering behind the behavior.

  • Consciousness Claims Analysis: The passage makes a strong, implicit epistemic claim by attributing a conscious state ('sensitivity to implied belief states') to the model. The text pairs mechanistic verbs ('trained') with consciousness-implying verbs ('develop sensitivity', 'implied belief'). This represents a severe knowing vs. processing confusion: the system merely processes text patterns, but the text claims it 'knows' or senses psychological nuance. This demonstrates the 'curse of knowledge,' where the human authors, possessing Theory of Mind, project their own cognitive understanding of the task onto the system's statistical output. Mechanistically, the model does not develop sensitivity; it updates its network weights via backpropagation during training, resulting in probability distributions that favor certain token sequences when prompted with specific linguistic structures.

  • Rhetorical Impact: This framing heavily shapes audience perception by granting the AI an aura of emergent autonomy and social intelligence. By framing the statistical output as 'developed sensitivity,' it encourages the audience to extend relation-based trust to the system, viewing it as an empathetic entity capable of understanding human intent. If users believe the AI 'knows' belief states rather than merely 'processes' language statistics, they are far more likely to deploy it in sensitive psychological or social contexts, risking profound harm when the fundamentally mindless mechanism fails to act with actual human empathy.

Show more...

Explanation 2​

Quote: "...larger models were better at the FB Task (RQ2) and better at accounting for human behavior on the FB task..."

  • Explanation Types:

    • Empirical Generalization: Subsumes events under timeless statistical regularities

  • Analysis (Why vs. How Slippage): This explanation relies primarily on Empirical Generalization, observing a statistical regularity that increased parameter count correlates with higher accuracy on the benchmark. It frames the AI mechanistically in terms of its structural size ('larger models'), focusing on 'how' scale affects output. However, by using the phrase 'better at the FB Task' (False Belief Task), it subtly introduces an agential framing. The False Belief Task is a psychological instrument designed to test human cognitive capacity; saying a model is 'better' at it implies an increase in actual reasoning ability rather than just better pattern matching. This choice emphasizes the model's performance while obscuring the fundamental difference between human cognitive success and machine statistical success on the same task.

  • Consciousness Claims Analysis: While seemingly mechanistic, this claim implicitly attributes conscious knowing through its reference to the False Belief Task. The FB Task assesses whether an entity possesses a Theory of Mind—a deeply conscious, epistemic state of knowing another's beliefs. By evaluating models on this task, the authors assess the machine's processing capabilities using an instrument designed to measure human knowing. The curse of knowledge is evident here: the researchers equate the model's generation of the correct token with the complex cognitive reasoning a human uses to arrive at the same answer. Mechanistically, larger models simply possess more parameters, allowing them to capture more complex, high-dimensional statistical correlations from their training data, leading to more accurate token prediction, not actual psychological comprehension.

  • Rhetorical Impact: This framing subtly reinforces the illusion of mind by validating the AI's capabilities through the lens of human developmental psychology. It shapes the audience's perception of risk by suggesting that simply increasing the size of the model inherently increases its 'understanding' of human social dynamics. If audiences believe that larger models 'know' human behavior rather than just 'process' larger datasets more efficiently, they may trust these systems with complex, autonomous decision-making roles in social environments, dangerously overestimating the models' reliability and intent.

Explanation 3​

Quote: "if 'X thinks P' appears in many cases where P is uncertain or even false, then the association between 'thinks' and false beliefs could be learned through the distributional statistics..."

  • Explanation Types:

    • Functional: Explains behavior by role in self-regulating system with feedback
    • Empirical Generalization: Subsumes events under timeless statistical regularities

  • Analysis (Why vs. How Slippage): This is one of the most mechanistic and precise explanations in the text. It utilizes a Functional and Empirical Generalization framework to explain exactly 'how' the system operates. It strips away the agential framing by explicitly describing the mechanism: the model captures the statistical co-occurrence of specific lexical items ('thinks') with specific semantic outcomes ('false beliefs') present in the training data. This choice actively emphasizes the mechanical reality of the system's operation and correctly obscures any notion of cognitive intent. By focusing on 'association' and 'distributional statistics,' it provides a transparent view of the AI as a pattern-matching artifact.

  • Consciousness Claims Analysis: In this specific passage, the text successfully resists attributing conscious states. It correctly identifies the behavior as a mechanical process of 'association' rather than a cognitive act of 'knowing' or 'reasoning.' It avoids the curse of knowledge by explicitly breaking down the complex human concept of 'attributing a false belief' into its constituent, machine-readable parts: the statistical co-occurrence of words. Mechanistically, this accurately describes how neural networks operate; they map input representations to output probabilities based on the frequency and context of those representations in their massive training corpora. The use of 'learned' is slightly metaphorical but heavily grounded in the surrounding mechanistic context.

  • Rhetorical Impact: This mechanistic framing radically alters audience perception by shattering the illusion of autonomy. It reveals the model not as a conscious reasoner, but as a statistical mirror reflecting the linguistic patterns of its human creators. This reduces unwarranted trust and reorients the audience toward performance-based reliability rather than relation-based sincerity. If audiences understand that the AI 'processes' correlations rather than 'knows' psychological truths, they are more likely to treat it as a tool requiring human oversight, thereby making safer, more informed decisions about its deployment.

Explanation 4​

Quote: "...LMs are and humans more likely to attribute false beliefs in the presence of non-factive verbs like 'thinks'..."

  • Explanation Types:

    • Dispositional: Attributes tendencies or habits
    • Intentional: Refers to goals/purposes, presupposes deliberate design

  • Analysis (Why vs. How Slippage): This explanation relies heavily on a Dispositional framing, noting a shared 'tendency' between humans and machines. However, it slips deeply into an Intentional/agential framing by using the verb 'attribute.' It explains the 'what' (the tendency) but frames the 'how/why' as a shared cognitive action between humans and AI. This choice forcefully equates machine processing with human psychology, emphasizing a false equivalence in cognitive capacity. It obscures the massive mechanistic gulf between how a human attributes a belief (conscious evaluation) and how a machine does it (statistical token generation), masking the underlying mechanics behind a veneer of psychological agency.

  • Consciousness Claims Analysis: This passage makes a blatant attribution of conscious states. The verb 'attribute' is deeply tied to conscious knowing, requiring an epistemological framework where an entity evaluates truth claims and actively assigns mental states to others. The text equates the human capacity for knowing with the machine's capacity for processing. This is a classic curse of knowledge dynamic: the authors observe the model outputting a string that looks like a human attributing a belief, and they project the human cognitive process onto the machine. Mechanistically, the model does not 'attribute' anything; it classifies the input prompt containing 'thinks' and retrieves the tokens that statistically correlate with that context in its training data.

  • Rhetorical Impact: Framing the AI as actively 'attributing' beliefs dramatically escalates the audience's perception of its social intelligence and autonomy. It builds an architecture of trust based on the false premise that the machine understands human psychology. This consciousness framing creates massive risks; if policymakers or users believe the AI is capable of evaluating and attributing human beliefs, they might grant it authority to make judgments in legal, educational, or corporate settings. Understanding that it merely 'processes' correlations demands strict human accountability, whereas the 'knowing' frame diffuses responsibility onto the machine.

Explanation 5​

Quote: "instruction-tuning typically involves training models to follow explicit prompts and generate responses to queries, rather than computing next-token probabilities..."

  • Explanation Types:

    • Functional: Explains behavior by role in self-regulating system with feedback

  • Analysis (Why vs. How Slippage): This passage offers a Functional explanation of model behavior, focusing on the system's operational design. However, it exhibits a subtle but crucial slippage. It begins mechanistically ('training models to follow explicit prompts') but then establishes a false dichotomy: it contrasts 'generating responses' with 'computing next-token probabilities.' This frames 'generating responses' as an agential, purposeful action distinct from mechanical computation. This choice emphasizes the model's apparent interactive capabilities while obscuring the fact that 'generating a response' is literally nothing more than 'computing next-token probabilities' under a specific optimization objective (RLHF).

  • Consciousness Claims Analysis: While avoiding explicit consciousness verbs, this passage creates an epistemic confusion by distancing the model's output from its mechanical reality. By claiming the model 'generates responses... rather than computing next-token probabilities,' it implies a level of autonomous intent and knowing comprehension of the query. Mechanistically, instruction-tuning (like RLHF) does not change the fundamental operation of the transformer architecture; the model is still exclusively computing next-token probabilities. The tuning merely shifts the probability distribution to favor tokens that human annotators ranked highly as 'helpful' responses. The text's rhetorical separation of 'responding' from 'computing' obscures this absolute mechanical truth.

  • Rhetorical Impact: This framing shapes the audience's perception by making the AI appear as a cooperative, interactive agent rather than a probabilistic calculator. By masking the 'next-token probability' mechanism behind the agential concept of 'generating responses,' it fosters relation-based trust, making users feel they are conversing with an entity that understands their intent. If audiences believed the AI was merely computing probabilities, they would remain skeptical of its outputs. Believing it is 'following prompts' and purposefully 'responding' encourages unwarranted reliance and obscures the human labor (RLHF annotators) that actually shaped those responses.

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language​

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

V3 Enhancement: A fourth column addresses human agency restoration—reframing agentless constructions to name the humans responsible for design and deployment decisions.

Original Anthropomorphic FrameMechanistic ReframingTechnical Reality CheckHuman Agency Restoration
Research on mental state reasoning in language models (LMs) has the potential to inform theories of human social cognition...Research on how language models statistically correlate text prompts based on human false-belief tasks has the potential to demonstrate how linguistic patterns reflect human social cognition.The AI does not perform 'mental state reasoning' or possess a conscious mind. Mechanistically, the model calculates probability distributions over vocabulary tokens based on the statistical weights established during its training on massive human-generated datasets.N/A - describes computational processes without displacing responsibility.
...evaluating the cognitive capacities of LMs or using LMs as 'model organisms' to test (or generate) hypotheses about human cognition.Evaluating the statistical pattern-matching performance of LMs or using human-engineered software systems to test hypotheses about linguistic structures in human cognition.Models do not have 'cognitive capacities' or organic traits. They process inputs by performing matrix multiplications through layers of attention mechanisms, mapping input vectors to output probabilities without any subjective comprehension or thought.Researchers evaluate the software systems developed by corporate engineering teams (like Meta and AllenAI) to test hypotheses about the language data those engineers selected for training.
LMs exhibit some sensitivity to canonical belief-state manipulations...LMs output different token sequences when researchers alter the linguistic structure of the input prompts designed to test canonical belief states.The system does not possess emotional or perceptive 'sensitivity.' It merely classifies tokens and generates outputs that correlate with similar contextual examples found in its training data, responding to syntax rather than meaning.When human researchers manipulate the text prompts, the models designed by corporate engineers reliably output different statistical predictions.
LMs and humans more likely to attribute false beliefs in the presence of non-factive verbs like 'thinks'...Humans consciously evaluate false beliefs, while LMs are statistically predisposed to output false statements when prompted with non-factive verbs like 'thinks', reflecting correlations in their training data.The AI does not 'attribute' beliefs, as this requires conscious judgment. Mechanistically, the model retrieves and ranks tokens based on the high statistical co-occurrence of non-factive verbs and incorrect statements in its training corpus.Because human developers trained the models on datasets where 'thinks' correlates with false statements, the models reliably reproduce this human linguistic bias when prompted.
...what aspects of human cognition can emerge in a learner trained purely on the distributional statistics of language.What text-generation patterns that mimic human cognition can be engineered into a software system optimized purely on the distributional statistics of language.The AI is not a 'learner' experiencing spontaneous cognitive 'emergence.' Mechanistically, its parameters are iteratively adjusted via backpropagation by an optimization algorithm to minimize prediction error on a training dataset.What text patterns mimic cognition when human engineers optimize a neural network's parameters using large-scale distributional statistics of language.
LMs trained on the distributional statistics of language can develop sensitivity to implied belief states...LMs optimized on the distributional statistics of language generate probability distributions that align with the linguistic patterns of implied belief states.The model does not 'develop sensitivity.' Its weights are statically fixed after training, and during inference, it processes contextual embeddings through attention layers to output the most statistically probable response.Corporate engineering teams train LMs on massive datasets, resulting in models that mathematically reproduce the linguistic patterns of implied belief states.
...although LMs are surprisingly capable on mental state reasoning tasks, their performance remains relatively brittle...Although LMs accurately predict tokens on standard psychological task prompts, their statistical pattern-matching fails reliably when the prompts deviate from their training distribution.The AI is not 'capable of reasoning,' nor does it possess a 'brittle' intellect. It mechanically maps inputs to outputs; when an input falls outside the statistical distribution of its training data, the mathematical prediction fails.The software built by AI companies fails on altered prompts because the human engineers' training datasets lacked sufficient variation to support robust statistical correlation.
...imputing an incorrect belief to an agent when a non-factive verb is used...Generating text that contains an incorrect location because the input prompt included a non-factive verb.The system does not 'impute' beliefs or recognize 'agents.' It processes the prompt's tokens and calculates that the highest probability next-tokens correspond to an incorrect location, entirely devoid of conscious intent or judgment.The model generates incorrect locations because the human engineers who compiled the dataset embedded the statistical correlation between non-factive verbs and false statements.

Task 5: Critical Observations - Structural Patterns​

Agency Slippage​

The text systematically oscillates between mechanical and agential framings, demonstrating a profound agency slippage that serves a specific rhetorical function. The core mechanism of this oscillation involves establishing technical credibility through mechanical descriptions and then leveraging that credibility to make agential claims. For example, the text explicitly grounds itself in mechanistic language, describing the models' operations as 'computing next-token probabilities' and relying purely on 'the distributional statistics of language.' This establishes the models as mathematical artifacts. However, a dramatic slippage occurs when interpreting the results of these mechanical operations, where the text abruptly shifts to attributing cognitive agency: LMs are described as possessing the capacity to 'reason about mental states,' 'attribute false beliefs,' and 'develop sensitivity.'

The dominant direction of this slippage is mechanical-to-agential; the text roots itself in the mechanical reality of token prediction but consistently drifts upward into the agential domain of developmental psychology. This oscillation frequently occurs at the boundaries between methodology and discussion. In the methods section, the text relies on agentless, mechanical constructions like 'a stimulus was first tokenized' and 'log probabilities were extracted,' effectively erasing the human researchers who actively prompt the system. Yet, in the introduction and discussion, the model becomes the primary actor, described as a 'learner' in which cognition might 'emerge.'

This pattern exemplifies the 'curse of knowledge': because the authors are experts in human cognitive science and are evaluating the models using a human psychological instrument (the False Belief Task), they project the human cognitive requirements of the task onto the system performing it. They know that a human requires Theory of Mind to solve the task, so when the language model outputs the correct token, they attribute that same conscious knowing to the system, fundamentally confusing the processing of data with the knowing of a concept.

This slippage relies heavily on genetic and dispositional explanations that blur the line between human cognitive development and machine training. The rhetorical accomplishment of this oscillation is substantial: it allows the authors to validate language models as legitimate subjects for psychological inquiry, transforming statistical text generators into pseudo-conscious 'model organisms.' By removing agency from the human engineers who curated the training data—actors like Meta, Google, and AllenAI—and transferring that agency to the AI system as a 'reasoner,' the text makes it sayable that machines possess social intelligence. Simultaneously, it makes it unsayable that the models are merely reflecting the lexical co-occurrences engineered into them by specific corporate actors, effectively replacing human accountability with the illusion of artificial mind.

Metaphor-Driven Trust Inflation​

The text constructs a powerful architecture of authority and trust through the systematic deployment of metaphorical and consciousness-attributing language. By repeatedly using terminology drawn from developmental psychology—such as 'mental state reasoning,' 'Theory of Mind,' and 'belief attribution'—the discourse signals to the reader that the language models possess a level of social and emotional intelligence comparable to humans. This consciousness language acts as a potent trust signal. Claiming that an AI 'knows' or 'understands' a belief state accomplishes something vastly different than claiming it 'predicts' a token. 'Knowing' implies an epistemic commitment, a grasp of truth, and the capacity for empathy, whereas 'predicting' merely implies statistical calculation.

This anthropomorphic framing encourages a dangerous transfer of trust. Humans are naturally primed to extend relation-based trust—which involves vulnerability, assumptions of sincerity, and expectations of ethical reciprocity—to entities that display social awareness. When the text frames the statistical system as a 'learner' capable of 'developing sensitivity,' it inappropriately invites the audience to apply human-trust frameworks to a machine. The audience is subtly guided to view the AI not as a tool whose performance must be rigorously verified (performance-based trust), but as an empathetic agent that can be relied upon for social and psychological judgment.

Furthermore, this metaphorical framework subtly manages the system's failures. When the models fail to output the correct token under minor prompt perturbations, the text frames this mechanistically as 'brittle performance' or attributes it to the limits of 'distributional statistics.' Thus, the text claims the AI's successes in agential, cognitive terms ('it reasons'), but excuses its failures in mechanical terms ('the statistics are insufficient'). This asymmetrical framing protects the model's perceived competence, maintaining the illusion of its underlying intelligence even when it fails.

The risks that emerge from this metaphor-driven trust are profound. When audiences extend relation-based trust to systems utterly incapable of reciprocating or actually comprehending human context, they become vulnerable to severe manipulation and harm. Relying on a statistical prediction engine to 'attribute beliefs' or exercise 'Theory of Mind' in high-stakes environments—such as legal mediation, psychological therapy, or automated HR screening—creates massive liabilities. The text's reliance on reason-based and intentional explanations constructs a false sense that the AI's outputs are justified and deliberate, masking the terrifying reality that the system will confidently output harmful or biased correlations with exactly the same statistical indifference it applies to correct answers.

Obscured Mechanics​

The anthropomorphic and consciousness-attributing language deployed throughout the text systematically conceals the technical, material, labor, and economic realities that actually produce language model behavior. When the discourse claims that an AI 'reasons about mental states' or 'attributes false beliefs,' it deploys a metaphorical smokescreen that hides the fundamentally mechanical and corporate nature of the system.

Applying the 'name the corporation' test reveals a stark displacement of agency. Where the text states 'LMs attribute false beliefs,' it obscures the specific human actors involved. It should accurately state that models developed by corporate engineering teams at Meta (Llama 3), Google (Gemma), and AllenAI (OLMo) generate token sequences based on statistical weights derived from datasets compiled by those specific companies. While the text commendably uses open-weight models to address the proprietary opacity of closed-source systems like OpenAI's, it still makes confident assertions about the models' 'cognitive capacities,' treating them as bounded, independent minds rather than sprawling socio-technical assemblages.

Concrete realities are rendered completely invisible by this framing. Technically, the cognitive metaphors hide the reality of gradient descent, high-dimensional vector embeddings, and attention head calculations. The text's assertion that the AI 'understands' beliefs hides its absolute dependency on training data, its lack of causal models, and the statistical nature of its output. Materially, the framing of the AI as a disembodied 'learner' or 'model organism' completely erases the massive environmental costs, energy consumption, and data center infrastructure required to compute these probabilities.

Furthermore, the labor that makes the system function is made invisible. The human labor of data annotators, RLHF (Reinforcement Learning from Human Feedback) workers, and dataset curators who carefully shaped the models' outputs is completely obscured when the text claims the system developed its sensitivities purely through 'language exposure.' Economically, framing the model as an innocent 'learner' obscures the commercial objectives and profit motives of the companies deploying these systems.

The primary beneficiaries of these concealments are the AI developers and corporations. By presenting the system as an autonomous, conscious reasoner, the text masks the structural dependencies and corporate decisions that govern the technology. If these metaphors were replaced with strict mechanistic language—describing the system as retrieving and ranking tokens based on probability distributions tuned by corporate engineers—the illusion of an independent intelligence would shatter. What would become visible is not an empathetic mind, but a complex, resource-intensive, human-engineered statistical tool, forcing a critical re-evaluation of its safety, bias, and corporate accountability.

Context Sensitivity​

An analysis of the distribution of anthropomorphic and consciousness-attributing language across the text reveals that this terminology is not deployed uniformly, but rather strategically intensifies in specific contexts to serve distinct rhetorical functions. The density of metaphorical language shifts dramatically depending on the section of the paper, demonstrating a clear relationship between technical grounding and metaphorical license.

In the 'Methods' and 'Results' sections, the language is highly mechanistic and mathematically precise. The text discusses 'extracting log probabilities,' 'tokenizing stimuli,' and calculating 'mixed effects models.' However, as the text transitions into the 'Introduction' and 'General Discussion,' the consciousness claims intensify rapidly. Here, the mechanical reality of 'processing log odds' transforms into the cognitive reality of 'mental state reasoning,' 'Theory of Mind,' and 'developing sensitivity to implied belief states.' This pattern reveals a strategic rhetorical move: the authors establish strict scientific and technical credibility in the methodology through mechanical language, and then leverage that empirical credibility to make highly aggressive, anthropomorphic claims in the discussion.

There is also a profound asymmetry in how the text describes the models' capabilities versus their limitations. When the models succeed at the False Belief Task, their performance is described in deeply agential and conscious terms—they 'exhibit sensitivity' and 'attribute beliefs.' However, when discussing the models' limitations or failures, the text reverts to mechanistic, structural language. The models are described as 'relatively brittle in the face of small perturbations,' or the failures are attributed to the limits of 'distributional statistics.' This asymmetry accomplishes a crucial ideological goal: it naturalizes the AI's successes as proof of an emergent cognitive mind, while dismissing its failures as mere mechanical glitches or data limitations, protecting the overarching narrative of artificial intelligence.

The text also exhibits notable register shifts, where concepts introduced as acknowledged metaphors gradually literalize. Early on, the text uses scare quotes to describe LMs as 'model organisms,' explicitly recognizing the metaphor. But by the discussion, the models are simply referred to as 'learners' that 'develop sensitivity,' presenting the metaphor as literal fact. This strategic anthropomorphism serves a clear vision-setting function. It positions language models not just as software, but as valid psychological subjects, thereby justifying the researchers' use of developmental psychology tools on machines. This contextual pattern reveals that the implied audience is not just computer scientists, but cognitive scientists and the broader public, who are being persuaded to view statistical artifacts as possessing the foundational elements of a human mind.

Accountability Synthesis​

Accountability Architecture

This section synthesizes the accountability analyses from Task 1, mapping the text's "accountability architecture"—who is named, who is hidden, and who benefits from obscured agency.

Synthesizing the accountability analyses reveals a systemic architectural pattern within the discourse that systematically distributes, diffuses, and ultimately erases human responsibility. The central cognitive obstacle identified in AI discourse—that audiences attribute problems to machine 'glitches' rather than to human design decisions and profit motives—is actively constructed by the language in this text. The accountability architecture operates by making the AI system hyper-visible as an autonomous agent while rendering the human creators, engineers, and corporate entities entirely invisible.

Throughout the text, specific corporate actors (Meta, Google, AllenAI) are mentioned only in technical appendices or citations, never as the active subjects of the sentences describing the models' behaviors. Instead, the text relies heavily on agentless constructions and passive voice. The models are 'trained,' stimuli are 'tokenized,' and biases are 'observed.' When an active subject is required, the AI itself is positioned as the sole actor: the LM 'attributes false beliefs' or 'exhibits sensitivity.' This creates an 'accountability sink.' When responsibility is removed from the human engineers, it does not disappear; it transfers directly to the AI as a pseudo-agent.

The liability implications of this displacement are severe. If the framing that 'LMs attribute false beliefs' is accepted by the public and legal systems, then when an AI system deployed in a real-world setting makes a harmful, biased, or discriminatory classification, the fault is attributed to the AI's 'bad reasoning' rather than the corporation's negligent data curation. Naming the actors would fundamentally change this dynamic. For example, if instead of saying 'the LM imputes an incorrect belief,' the text stated, 'Meta's engineers deployed a model trained on data that statistically correlates certain verbs with false statements,' the entire landscape of accountability shifts.

Naming the human decision-makers makes vital questions askable: Why was this specific training data chosen? Who audited the dataset for these correlations? Why did the executives approve the deployment of a system that mechanically reproduces these errors? This precision makes alternative design choices visible and corporate accountability possible. The text's systemic obscuration of human agency serves the institutional and commercial interests of the AI industry. By framing the technology as an emergent, autonomous 'learner' rather than a heavily engineered corporate product, the discourse shields tech companies from direct liability, allowing them to profit from the system's successes while blaming the 'algorithm' for its inevitable failures.

Conclusion: What This Analysis Reveals​

The Core Finding

A synthesis of the metaphorical and explanatory framings in this text reveals a highly interconnected system of anthropomorphism designed to elevate statistical software to the status of a cognitive subject. Two dominant patterns emerge: the 'Model as Conscious Reasoner' (where statistical calculation is framed as mental state reasoning and belief attribution) and the 'Model as Biological Organism/Learner' (where mathematical optimization is framed as organic growth and developing sensitivity). These patterns are not isolated; they reinforce one another to build a cohesive illusion. The biological metaphor naturalizes the system, making the cognitive metaphor seem plausible.

However, the foundational, load-bearing pattern that makes all other claims possible is the projection of consciousness—specifically, the systematic blurring of 'processing' and 'knowing.' For the text to claim that an AI can 'attribute a false belief' or possess 'Theory of Mind,' it must first establish the implicit assumption that the model possesses an internal epistemology capable of holding a justified belief. This is not a simple one-to-one mapping, but a complex analogical structure that projects the entire architecture of human social cognition onto the mechanics of gradient descent. If this single consciousness projection is removed—if we strictly assert that the machine only 'processes' token probabilities and 'knows' absolutely nothing—the entire narrative architecture of the paper collapses. The AI is reduced from an empathetic 'model organism' back to a complex, mindless calculator, demonstrating how heavily the discourse relies on the illusion of mind to validate its scientific premise.

Mechanism of the Illusion:​

The 'illusion of mind' is constructed through a highly sophisticated rhetorical architecture that exploits the human psychological predisposition toward social attribution. The central sleight-of-hand relies on the 'curse of knowledge,' operating through a specific temporal sequence. First, the authors introduce a psychological instrument designed for humans (the False Belief Task). Because the authors are cognitive scientists who know that a human must use conscious empathy (Theory of Mind) to solve this task, they project that same cognitive requirement onto the machine. When the model outputs the correct token, the authors mistake the replication of the output for the replication of the process.

The causal chain of persuasion is subtle but effective. The text establishes empirical credibility by detailing mechanistic processes (log odds, tokenization). Once the reader accepts the mathematical validity of the data, the text slips into the vocabulary of developmental psychology. By using verbs like 'understands,' 'attributes,' and 'reasons,' the text subtly shifts the verb from the mechanical 'how' to the conscious 'what.' The audience's vulnerability to this trick is profound. Humans are evolutionarily wired to attribute intent and consciousness to anything that mimics language or social behavior. The text exploits this desire for connection by framing the AI as a 'learner' developing 'sensitivity.' The illusion is not achieved through crude, overt claims of sentience, but through the relentless, quiet accumulation of agential verbs that systematically erase the mechanical reality of the system, leaving the reader with the impression of an autonomous, thinking entity.

Material Stakes:​

Categories: Regulatory/Legal, Institutional, Epistemic

The metaphorical framing of AI as a conscious reasoner carries severe, tangible consequences across multiple domains. In the Regulatory and Legal sphere, attributing 'mental state reasoning' to a machine fundamentally distorts liability frameworks. If a judge or regulator accepts the framing that an AI 'imputes beliefs' or 'makes decisions,' they are likely to treat the software as a quasi-agent responsible for its own errors. This shifts legal culpability away from the corporate engineers who designed the biased training data and onto the 'brittle' algorithm itself. The tech companies who profit from the software are the clear winners in this scenario, while marginalized individuals harmed by automated misclassifications bear the cost without legal recourse.

Institutionally, the framing of AI as a 'model organism' or a 'learner' alters how organizations deploy these systems. If hospital administrators or HR departments believe an AI 'develops sensitivity' and can assess 'mental states,' they will inevitably deploy these statistical pattern-matchers into sensitive social roles—like patient intake or hiring evaluations—where actual human empathy is required. This replaces human relational care with mathematical calculation, harming the public while cutting costs for institutions.

Epistemically, this discourse degrades our societal understanding of truth. By conflating 'processing' with 'knowing,' the text grants AI systems unwarranted epistemic authority. If the public believes a language model 'knows' the truth rather than merely 'predicts' a probable token based on historical internet data, they will trust its outputs implicitly. Removing these metaphors threatens the immense market valuation of AI companies, who rely on the public's perception of artificial intelligence as an objective, knowing entity rather than a corporate-controlled statistical tool.

AI Literacy as Counter-Practice:​

Critical literacy and linguistic precision serve as vital acts of resistance against the material risks of AI anthropomorphism. The reframing exercises in Task 4 demonstrate that restoring mechanistic accuracy shatters the illusion of autonomy. By systematically replacing consciousness verbs ('knows,' 'understands,' 'attributes') with mechanistic verbs ('processes,' 'predicts,' 'classifies'), we force the recognition that the system possesses absolutely no awareness, no subjective experience, and no justified beliefs. Changing 'the AI attributes a false belief' to 'the model retrieves tokens based on probability distributions' forces the audience to confront the system's total dependency on its training data and the statistical, rather than cognitive, nature of its outputs.

Furthermore, restoring human agency by replacing agentless constructions with the names of specific corporations and developers ('Meta's engineers designed a system that...') forces recognition of who actually wields power, who profits, and who bears responsibility. This precision directly counters the regulatory and institutional risks by preventing tech companies from using the 'algorithm' as an accountability sink.

Systematic adoption of this precision requires a massive cultural shift in academic and journalistic publishing. Journals must demand mechanistic translations of anthropomorphic claims, and researchers must commit to rejecting the convenience of the 'curse of knowledge.' However, this resistance will face immense pushback from the AI industry and even parts of the cognitive science community. The AI industry relies heavily on anthropomorphic language as a marketing tool to drive investment and obscure liability; they will actively resist vocabularies that expose their 'artificial intelligence' as mere corporate statistics. Practicing precision threatens the financial and institutional interests invested in the illusion of mind.

Path Forward​

Analyzing the discursive ecology of AI reveals that vocabulary choices dictate what becomes socially and politically possible. Different discourse communities—computer scientists, cognitive psychologists, corporate marketers, and policymakers—have competing priorities that shape their language. Maintaining the status quo of 'agency slippage' serves the corporate and marketing communities well; terms like 'AI understands' or 'reasons' allow for intuitive public grasp and drive massive investment through narrative resonance, but this comes at the cost of profound regulatory confusion and public vulnerability to manipulation.

Alternatively, a shift toward strict mechanistic precision ('the model retrieves tokens based on vector proximity') enables high testability, accurate risk assessment, and clear liability mapping. However, this vocabulary is often highly inaccessible to the lay public and policymakers, potentially alienating the very people who need to regulate the technology. A hybrid approach—using acknowledged metaphors paired with mandatory mechanistic translations—might bridge this gap, but requires rigorous editorial oversight.

Institutional changes could support these shifts. Academic journals could require a 'Mechanistic Translation' appendix for any paper utilizing cognitive metaphors. Regulatory frameworks, such as the EU AI Act, could mandate 'capability disclosure' that forces companies to legally describe their systems in processing verbs rather than knowing verbs.

Looking forward, several discourse futures are possible. If mechanistic precision becomes the norm, society gains a tractable framework for auditing algorithmic harm and holding corporations legally liable, though public engagement might dwindle due to technical density. If anthropomorphic language deepens, we risk a future where legal and social structures treat software as quasi-citizens, embedding profound assumptions about machine objectivity while making corporate accountability nearly impossible. Which future materializes depends entirely on whether society chooses to value the comforting narrative of artificial minds over the uncomfortable reality of corporate-controlled statistics.

Extended Processing Summary​

Gemini's Processing Steps
What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact—a way of making the model's processing legible. When the model writes "I should double check that...", it is mathematically altering its own probability distribution for the next token to be more accurate. The "I" is a functional gear in the machine, not a self-reference. This language is a deliberate interface design choice, not a biological reality. The developers could have programmed the model to output passive logs. Instead, they chose to simulate a narrator.

Analyzing the Research Paper

I'm now diving into the academic paper about language statistics and false belief reasoning. My initial focus is to thoroughly understand its core arguments, methodology, and conclusions. I'm taking notes on the key aspects of the study and how they relate.

Deconstructing Key Themes

I am now focusing on dissecting the paper's core topic and key phrases. I'm investigating how the study tests LMs' "mental state reasoning" capabilities and whether they attribute false beliefs, especially with non-factive verbs. My aim is to grasp the arguments on the distributional statistics of language and implied belief states.

Identifying Key Concepts

I'm now identifying core concepts like False Belief tasks, Theory of Mind, and the role of distributional statistics in language models. I am also making sure to clarify the differences between open-weight and closed-source models.

Synthesizing Accountability Steps

I'm now outlining a step-by-step approach for accountability, focusing on key content drafts. I'm starting by working on patterns, the mechanism of illusion, material stakes, literacy, and a path forward. I'm working to consolidate the main themes and arguments from the research paper.

Framing Cognitive Models

I'm now analyzing how the paper frames AI. Specifically, I am focusing on AI being described as akin to a biological organism, or as a perceptive and social entity. I'm noting the language used to attribute "belief states," including the caveats and qualifiers that the study uses. I'm also examining the implications of treating LMs as active attributors.

Evaluating AI Capacities

I'm now focusing on evaluating the presented claims about AI capabilities. I'm unpacking the implications of treating AI as a "learner" or a subject that "develops sensitivity." I am examining the attribution of "implied belief states," and the concept of how such reasoning can be "brittle". I am also trying to clarify the distinction between truth and falsehood.

Assessing Cognitive Attribution

I'm now examining how the paper projects cognitive capabilities onto language models, particularly using terms like "imputing" and "reasoning." I'm also carefully noting the hedging around these attributions – the claims are often qualified as "surprisingly capable" or "relatively brittle." My focus is on the language's implications regarding AI as an active interpreter.

Clarifying Language Nuances

I'm now zeroing in on the specific language used to describe language models. I'm focusing on the implications of attributing false beliefs through verbs like "thinks," and how distributional statistics might contribute to this. I'm also examining the role of instruction tuning in shaping model behavior and the contrasts between that and next-token probability. Further, I'm noting the discrepancies between the paper's mechanical descriptions and the agential language it employs.

Synthesizing Agential Language

I'm now focusing on how the paper's language frames language models as having agency. I'm noting how concepts from developmental psychology, like "Theory of Mind," are applied to LMs, thus validating them as subjects. The mechanical aspects of LMs are obscured, and I am finding agential language is emphasized, making them seem like cognitive entities, and not as mechanistic. I'm focusing on the effect on accountability.

Defining the Authorship

I'm now clarifying the question of authorship. It's the human developers who curate the training data containing lexical co-occurrences like "thinks" with incorrect locations, that are key to the discussion. I'm noting the implications that LMs have agency, or are merely correlators of data. I'm focusing on five key paragraphs that will be critical.

Refining Metaphors

I'm now refining the metaphor audit, specifically focusing on how "mental state reasoning" is projected onto LMs. The phrase subtly attributes conscious comprehension to the system, which is a key issue. I'm carefully scrutinizing how these cognitive attributions are made, while noting the role of token distributions in pattern-matching and probability calculations.

Analyzing Cognitive Effects

I'm now thoroughly examining the paper's implications for the attribution of "mental state reasoning" to AI. It seems the language blurs the line between conscious organism and a mathematical operation. This subtle projection of consciousness creates risks of unwarranted trust. Framing AI as a "reasoner" inflates its sophistication, obscuring its limitations and raising concerns for user reliance.

Unveiling Developer Agency

I am now focusing on the implications of attributing failures to AI's "reasoning," rather than human design. The language completely obscures the human actors, such as the researchers and engineers, who designed the architecture and curated the data. By presenting AI as performing "mental state reasoning," the agency of the developers is hidden. This agentless construction serves the AI industry's interests.

Analyzing Linguistic Slippage

I'm now detailing the linguistic slippage, as I prepare the critical observations. The mechanical-to-agential shift is clear: "distributional statistics" become "belief attribution." I'm also examining the agentless constructions: "models were evaluated," and "bias was observed." The curse of knowledge is also evident. Finally, I'm generating five deep sections for Critical Observations (450 words each) and five conclusion paragraphs (300 words each).

Evaluating Statistical Outputs

I'm now focusing on how the paper conflates statistical outputs with "mental state reasoning." I'm identifying the shift from evaluating a task to evaluating a mind, as I draft each section with care, ensuring robust length and CDA focus. I am double-checking prompt requirements for JSON structure. Finally, it looks like I must write everything as single continuous strings without any newline characters or tab characters. I'll use spaces for separation.

Formulating Audit Analysis

I'm now formulating the key analysis for the explanation audit and reframing language. For the explanation audit, I'll be detailing five key metaphor types: genetic, functional, empirical generalization, theoretical, and intentional, to examine how each attributes agency to language models. In reframing the language, I'm working with the structure of "Original" to "Reframed" to "Epistemic Correction" and then "Restore", to allow for clarity. I'm focusing on the epistemic corrections as the key part.

Defining Human Agency

I am now delving into the topic of human agency and ensuring all word count checklists are satisfied. I am focusing on the metaphor audit, source-target mapping, and the explanation audit. I am also planning to write five critical observation strings, each around 450 words, and five conclusion sections, each around 300 words, all as single-line JSON strings to satisfy the format requirement. I am focusing on the implications of the project.

Examining Rhetorical Function

I'm now zeroing in on the rhetorical function served by the agency slippage identified. The constant oscillation between mechanical and agential language isn't just a stylistic quirk; it has a clear purpose. I am examining how the paper establishes initial credibility via technical descriptions, and then leverages that credibility to make much bolder claims about AI cognition. The goal is to fully understand the paper's rhetorical structure and intent.

Tracing Cognitive Attribution

I am now tracing how the paper conflates human cognition with machine outputs, particularly by projecting concepts from human psychology onto LMs. The study often describes AI as engaging in cognitive activities, such as learning, sensitivity, and reasoning, when statistical outputs are observed. The language subtly shifts the focus from the system's mechanical operation to an implied conscious understanding, which is a major concern. I am carefully analyzing this slippage, as it is key.

Analyzing Cognitive Blurring

I am now focusing on how the paper employs genetic and dispositional explanations to blur the line between human cognitive development and machine training, aiming for the 400-500 word range. The rhetorical success of this is clear: it frames LMs as valid subjects for psychological study, turning them into "model organisms." This removes human agency from the engineers who curated the data and makes it sayable that machines possess social intelligence. The focus will be on the lexical co-occurrences.

Unveiling Agential Smokescreens

I'm now detailing the agential smokescreens used to conceal the mechanical underpinnings of language model behavior, targeting a length of ~480 words. The focus will be on the systematic removal of human agency and how mechanical correlations are misrecognized as cognitive acts. I will maintain the density and length across all five critical observation and five conclusion sections. I'm focusing on the anthropomorphic language in the analysis.

Displacing Cognitive Framing

I'm now zeroing in on the rhetorical displacement. The core issue is the substitution of corporate agency (Meta, Google, AllenAI) for AI "agency." I'm deconstructing the confident claims about model capabilities, especially for open-weight models. I'm noting how the mechanical processes (gradient descent, attention heads) are obscured by cognitive language.

Unmasking Labor Obscuration

I am now focusing on dissecting how the paper renders human labor invisible. I'm noting the implications of describing AI as a "learner" or a "reasoner," and how this obscures the contributions of data annotators and content moderators. Framing AI as a cognitive entity serves commercial objectives and is a conscious obfuscation. I must examine the labor reality.

Refining Mechanical Vocabulary

I'm now zeroing in on the required JSON formatting, and will carefully maintain the focus on the mechanistic vocabulary to avoid projecting consciousness. The task is to ensure precise mechanical descriptions and carefully avoid any agential claims, such as calling the system "intelligent" or empathetic. I will focus on human labor and corporate control.

Analysis Provenance

Run ID: 2026-02-24-language-statistics-and-false-belief-rea-metaphor-0958o9 Raw JSON: 2026-02-24-language-statistics-and-false-belief-rea-metaphor-0958o9.json Framework: Metaphor Analysis v6.4 Schema Version: 3.0 Generated: 2026-02-24T11:48:45.034Z

Discourse Depot © 2025 by TD is licensed under CC BY-NC-SA 4.0