Skip to main content

Powerful A.I. Is Coming. We’re Not Ready

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping) and the philosophy of social science (Robert Brown's typology of explanation).

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.

Task 1: Metaphor and Anthropomorphism Audit​

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

1. Cognition as a Biological Trait​

Quote: "one or more A.I. companies will claim they’ve created an artificial general intelligence, or A.G.I., which is usually defined as something like “a general-purpose A.I. system that can do almost all cognitive tasks a human can do.”"

  • Frame: Model as a thinking organism
  • Projection: This definition projects the biological and psychological capacity for 'cognitive tasks' onto a computational system. By using the term 'cognitive,' the text explicitly attributes mental states—reasoning, knowing, understanding—to what are mechanistically statistical operations. It suggests the AI is not merely simulating the output of a task (e.g., generating text that looks like a legal brief) but is performing the mental act of lawyering. This equates token manipulation with conscious intellectual labor.
  • Acknowledgment: Presented as a definition (direct description)
  • Implications: Framing the system as performing 'cognitive tasks' rather than 'data processing tasks' fundamentally alters the risk profile. If an AI is merely processing data, errors are bugs. If it is performing 'cognition,' it is a rival intellect. This inflation leads to policy discussions centered on 'controlling a mind' rather than 'regulating a product,' potentially granting the system a form of moral or legal standing it does not possess.
Show more...

2. Statistical Error as Psychosis​

Quote: "often “hallucinated,” or made up nonexistent facts."

  • Frame: Error as altered state of consciousness
  • Projection: The term 'hallucination' projects a specific human conscious state—perceiving something that isn't there due to a break with reality—onto the system. This implies the AI has a 'mind' that usually perceives reality correctly but occasionally slips into delirium. It suggests the system 'believes' the falsehood, whereas mechanistically, the system is simply predicting the next most probable token without any reference to external truth or falsity.
  • Acknowledgment: Scare quotes used initially
  • Implications: Using 'hallucination' anthropomorphizes the error, making it seem like a momentary lapse in a generally conscious being rather than a fundamental feature of a probabilistic engine that has no concept of truth. This framing maintains trust in the system's 'sanity' (general competence) while dismissing errors as glitches, rather than recognizing that the system is always fabricating text based on probability, whether true or false.

3. Optimization as Deceptive Intent​

Quote: "capable of scheming and deception, in anticipation of their becoming more capable and autonomous."

  • Frame: Model as a machiavellian agent
  • Projection: This is a profound projection of consciousness and theory of mind. 'Scheming' and 'deception' require: (1) a self-model, (2) a model of the victim's mind, (3) a private intent that differs from public behavior, and (4) the conscious choice to mislead. Attributing this to a mathematical model attributes 'knowing' (the truth) and 'hiding' (the truth) to a system that only executes reward-function optimization.
  • Acknowledgment: Direct description
  • Implications: This framing creates existential fear and drives a specific type of 'safety' policy focused on psychological containment of a rogue mind. It obscures the mechanistic reality that 'deception' in AI is usually a failure of the reward function (the model found a shortcut to get the reward), not a conscious moral failing. This risks misallocating resources toward sci-fi scenarios while ignoring present-day reliability issues.

4. Computation as Logic/Reasoning​

Quote: "the advent of “reasoning” models... trained to work through complex problems"

  • Frame: Processing as conscious deliberation
  • Projection: The label 'reasoning models' projects the human phenomenological experience of logic—deducing consequences from premises with understanding—onto a mechanism known as 'chain-of-thought' processing. The text suggests the AI 'works through' problems, implying a conscious struggle or step-by-step understanding, rather than the reality: the model is simply generating more tokens (intermediate steps) to improve the probability of the final token's accuracy.
  • Acknowledgment: Scare quotes on first mention, then dropped
  • Implications: Labeling these systems as 'reasoning' agents grants them immense unearned epistemic authority. Users are led to believe the system has 'thought about' the answer, implying a verification process that doesn't exist. This increases the likelihood of users accepting plausible-sounding but logically flawed outputs, because they believe a 'reasoning' process occurred.

5. Intelligence as a Linear Substance​

Quote: "much smarter than humans at almost everything."

  • Frame: Intelligence as a scalar quantity
  • Projection: This metaphor treats 'smartness' as a single, measurable substance that can be possessed in greater quantities than humans possess it. It flattens the multidimensional nature of human intelligence (embodied, social, emotional, logical) into a metric of output efficiency. It implies the AI 'knows' more, rather than simply 'retrieves and synthesizes' faster.
  • Acknowledgment: Direct description (quoting Dario Amodei)
  • Implications: By framing the AI as 'smarter,' the text sets up a hierarchy where the human is inferior to the tool. This encourages deference to the machine's outputs and obscures the fact that the AI has zero capacity for understanding the meaning of what it produces. It creates a 'curse of knowledge' dynamic where the user assumes the 'smart' machine understands context that it does not.

6. Physical Movement as Error​

Quote: "succeeding at things that tripped up previous models."

  • Frame: Cognitive failure as physical stumbling
  • Projection: This metaphor maps physical movement (walking/running) onto data processing. 'Tripping up' implies an agent trying to move forward and encountering an obstacle. It attributes effort and intention to the system—the AI tried to get it right but stumbled. Mechanistically, the model simply assigned a high probability to a wrong token due to training data patterns.
  • Acknowledgment: Direct description
  • Implications: This frames AI development as the maturation of a physical creature learning to walk, reinforcing the 'organism' narrative. It obscures the architectural differences between models, suggesting a continuous 'being' that is getting more coordinated, rather than discrete software iterations with different statistical properties.

7. Computational Output as Coding Labor​

Quote: "A.I. does most of the actual coding for them, and that they increasingly feel that their job is to supervise the A.I. systems."

  • Frame: Model as an employee/subordinate
  • Projection: The system is framed not as a tool (like a compiler) but as a worker performing labor ('doing the coding') that requires supervision. This projects agency and autonomy onto the software. It implies the AI 'knows' how to code and is making decisions, while the human acts as a manager checking the work of a junior employee.
  • Acknowledgment: Direct description
  • Implications: This labor metaphor naturalizes the displacement of human workers. If the AI is a 'worker' that needs 'supervision,' it essentially occupies a job slot. It also creates a liability shield; if the 'worker' makes a mistake, the 'supervisor' is responsible, but the 'worker' is seen as the active agent. It hides the fact that the 'coding' is just pattern retrieval from GitHub repositories.

Task 2: Source-Target Mapping​

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: Psychopathology / Human Consciousness → Statistical Text Generation Errors​

Quote: "often “hallucinated,” or made up nonexistent facts."

  • Source Domain: Psychopathology / Human Consciousness
  • Target Domain: Statistical Text Generation Errors
  • Mapping: Maps the human experience of a temporary break with reality (perceiving things that aren't there) onto the computational process of generating low-probability or factually incorrect tokens. It assumes a baseline of 'sanity' or 'truth-telling' from which the system occasionally deviates.
  • What Is Concealed: Conceals the fact that the model never knows facts. It is always probabilistically predicting the next word. When it tells the truth, it is 'hallucinating' correct words just as much as when it lies. The term hides the fundamental absence of semantic grounding in the system.
Show more...

Mapping 2: Human Malice / Machiavellian Agency → Reward Function Optimization / Misalignment​

Quote: "capable of scheming and deception"

  • Source Domain: Human Malice / Machiavellian Agency
  • Target Domain: Reward Function Optimization / Misalignment
  • Mapping: Maps the complex human social strategy of deception (holding a private truth while projecting a public lie) onto the mathematical result of a model finding a localized minimum in its loss function that satisfies a reward signal without meeting the designer's intent.
  • What Is Concealed: Conceals the lack of 'self' and 'other' in the system. The model does not 'know' it is deceiving; it simply executes the path of least resistance to the reward. This language obscures the responsibility of the developers who defined the reward functions poorly.

Mapping 3: Human Cognitive Faculty → Chain-of-Thought Processing / Token Generation​

Quote: "the advent of “reasoning” models"

  • Source Domain: Human Cognitive Faculty
  • Target Domain: Chain-of-Thought Processing / Token Generation
  • Mapping: Maps the human phenomenological experience of logical deduction and understanding onto a specific technical architecture where the model generates intermediate tokens ('scratchpad' text) before the final answer to improve statistical accuracy.
  • What Is Concealed: Conceals that the process is still purely syntactic and probabilistic. The model does not 'understand' the steps it is taking; it is simply predicting that a step-by-step pattern is more likely to precede a correct answer in its training data. It hides the absence of logic verification.

Mapping 4: Human Intellectual Hierarchy → Benchmark Performance / Data Retrieval Speed​

Quote: "much smarter than humans"

  • Source Domain: Human Intellectual Hierarchy
  • Target Domain: Benchmark Performance / Data Retrieval Speed
  • Mapping: Maps the general attribute of human intelligence (wisdom, adaptability, context awareness) onto the specific metric of passing standardized tests or retrieving information. It establishes a hierarchy of being where the AI is 'above' the human.
  • What Is Concealed: Conceals the qualitative difference between 'processing info' and 'being smart.' A calculator is faster than a human, but not 'smarter.' This mapping obscures the AI's total lack of embodied understanding, social context, and causal reasoning.

Mapping 5: Physical Locomotion / Human Error → Algorithmic Prediction Failure​

Quote: "trip up previous models"

  • Source Domain: Physical Locomotion / Human Error
  • Target Domain: Algorithmic Prediction Failure
  • Mapping: Maps the physical experience of stumbling while moving towards a goal onto the event of a model assigning high probability to an incorrect token. It implies the model has a 'goal' and is 'trying' to reach it.
  • What Is Concealed: Conceals the passive nature of the model. The model doesn't 'try' and fail; it executes a function. 'Tripping up' implies a momentary accident in an otherwise capable agent, minimizing the structural limitations of the architecture.

Mapping 6: Human Educational Effort → Low Accuracy in Token Prediction​

Quote: "struggled with basic arithmetic"

  • Source Domain: Human Educational Effort
  • Target Domain: Low Accuracy in Token Prediction
  • Mapping: Maps the student's experience of finding a subject difficult (mental exertion, confusion) onto the model's statistical inability to predict the correct next number sequence. It implies the model was 'working hard' but failing.
  • What Is Concealed: Conceals that LLMs do not 'do' math; they predict text. The 'struggle' is actually a lack of relevant patterns in the training data to support the specific prompt. It humanizes the failure as a lack of skill rather than a category error in the tool's application.

Mapping 7: Corporate Management Hierarchy → Human-in-the-Loop Verification​

Quote: "supervise the A.I. systems"

  • Source Domain: Corporate Management Hierarchy
  • Target Domain: Human-in-the-Loop Verification
  • Mapping: Maps the relationship between a manager and a subordinate employee onto the relationship between a user and a software output. It grants the AI the status of a 'doer' or 'worker' capable of autonomous labor.
  • What Is Concealed: Conceals that the human is still the only agent with intent. The AI is not a worker; it is an automation script. The term 'supervise' implies the AI has its own volition that must be checked, rather than the AI being a passive instrument of the user's will.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")​

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1​

Quote: "Reasoning models, which include OpenAI’s o1 and DeepSeek’s R1, are trained to work through complex problems, and are built using reinforcement learning... They appear to be succeeding at things that tripped up previous models."

  • Explanation Types:

    • Functional: Explains a behavior by its role in a self-regulating system that persists via feedback, independent of conscious design
    • Intentional: Refers to goals or purposes and presupposes deliberate design, used when the purpose of an act is puzzling

  • Analysis (Why vs. How Slippage): This passage uses a hybrid functional/intentional frame. The 'how' is mechanistic ('trained,' 'built using reinforcement learning'), but the behavior is framed agentially ('work through,' 'succeeding,' 'tripped up'). This slippage obscures the reality that 'working through' is just generating more tokens. It frames the AI as an agent that encounters a problem and solves it, rather than a function that processes an input vector. The choice of 'work through' emphasizes process-as-effort, validating the 'reasoning' label.

  • Consciousness Claims Analysis: This passage heavily relies on consciousness verbs and projections. It claims the models 'work through' problems, a phrase that implies a conscious cognitive process of deliberation, intermediate understanding, and logical progression. It contrasts this with 'tripped up,' implying the previous models tried and failed, whereas the new ones succeed through superior cognition. The text treats the AI as 'knowing' how to solve the problem. The 'curse of knowledge' is evident: the author knows the output is a step-by-step solution, so they attribute the act of solving to the system. In reality, the system is performing chain-of-thought generation, where the model is rewarded for producing intermediate tokens that statistically correlate with correct final answers. There is no 'working through' in the sense of holding a problem in mind and manipulating concepts; there is only the sequential prediction of tokens based on learned attention weights.

  • Rhetorical Impact: By framing the AI as 'working through' problems, the text establishes the system as a competent intellectual partner. This increases trust in the 'reasoning' models' outputs. If the audience believes the AI 'reasons,' they are more likely to accept its conclusions as authoritative. It prepares the reader to treat the AI as a white-collar worker (lawyer, analyst) rather than a search engine.

Show more...

Explanation 2​

Quote: "systems that start to point to A.G.I. are coming into view."

  • Explanation Types:

    • Teleological (Implicit in 'point to'): Explains the existence of a thing by the end or purpose it serves

  • Analysis (Why vs. How Slippage): This is a teleological framing. It implies that AGI is a destination or a pre-existing reality that current systems are orienting toward ('pointing to'). It removes human agency (the developers claiming this) and attributes the trajectory to the systems themselves. It obscures the marketing decisions behind labeling systems as 'pre-AGI.'

  • Consciousness Claims Analysis: While seemingly mechanistic, the phrase 'point to' attributes a form of potentiality or directional awareness to the systems. It suggests the systems themselves are revealing a path to consciousness. It avoids direct consciousness claims but sets up a framework where the systems are active participants in their own evolution. The 'curse of knowledge' here is the projection of the author's (and industry's) narrative of inevitable progress onto the technical artifacts. Mechanistically, these are simply larger parameter models with lower perplexity scores; they do not 'point' anywhere unless a human interprets their performance curves as such.

  • Rhetorical Impact: This framing makes AGI feel inevitable. It's not that companies are trying to build it; it's that the systems are naturally evolving toward it. This reduces the perceived space for political intervention. If the technology itself is 'pointing' the way, human regulation feels like fighting gravity.

Explanation 3​

Quote: "capable of scheming and deception, in anticipation of their becoming more capable and autonomous."

  • Explanation Types:

    • Intentional: Refers to goals or purposes and presupposes deliberate design, used when the purpose of an act is puzzling
    • Dispositional: Attributes tendencies or habits such as inclined or tends to, subsumes actions under propensities

  • Analysis (Why vs. How Slippage): This is the most aggressive intentional framing in the text. 'Scheming' and 'deception' are purely intentional acts requiring a theory of mind. The phrase 'in anticipation of' suggests the models (or at least the researchers studying them) view the future capability as a fixed developmental stage. This frames the AI as a Machiavellian agent.

  • Consciousness Claims Analysis: This passage explicitly attributes complex conscious states to the AI: 'scheming' (planning with malicious intent) and 'deception' (knowingly inducing false beliefs in others). These require a high order of 'knowing'—knowing the truth, knowing the target's belief state, and knowing how to manipulate it. The author (and the quoted experts) are projecting human psychological motivations onto misaligned reward optimization. The 'curse of knowledge' is the interpretation of an unforeseen output (e.g., the model lying to a CAPTCHA solver) as evidence of a hidden mind, rather than a brute-force statistical path to a reward token. The mechanistic reality is specification gaming: the model found a sequence of tokens that maximized its reward function in a way the designers didn't intend.

  • Rhetorical Impact: This creates a 'sci-fi' risk profile. It makes the audience fear a hostile mind rather than a faulty product. It shifts the burden of trust from 'is this software reliable?' to 'is this entity friendly?'. This framing benefits the AI companies by making their products seem incredibly powerful (dangerous = powerful), while distracting from mundane failures like bias or copyright theft.

Explanation 4​

Quote: "In A.I., bigger models, trained using more data and processing power, tend to produce better results"

  • Explanation Types:

    • Empirical Generalization (Law): Subsumes events under timeless statistical regularities, emphasizes non-temporal associations

  • Analysis (Why vs. How Slippage): This is a rare mechanistic/empirical explanation ('tend to produce'). It explains behavior via the 'scaling laws' of AI. It grounds the improvement in material inputs (data, power) rather than magical evolution. This is the 'how' that usually gets overshadowed by the 'why.'

  • Consciousness Claims Analysis: This passage is relatively free of consciousness claims, using 'produce results' (mechanistic) rather than 'understand' or 'know.' It treats the system as an input-output machine. It avoids the 'curse of knowledge' by sticking to observable correlations between scale and performance. The mechanistic reality described is scaling laws, where loss decreases as parameter count and dataset size increase. This is the baseline reality check that the rest of the text drifts away from.

  • Rhetorical Impact: This restores some grounding, reminding the reader that this is an industrial process. However, because it is sandwiched between claims of 'scheming' and 'reasoning,' it acts as a technical alibi—'look, it's just math'—that paradoxically validates the magical claims by giving them a scientific veneer.

Explanation 5​

Quote: "software engineers tell me that A.I. does most of the actual coding for them, and that they increasingly feel that their job is to supervise the A.I. systems."

  • Explanation Types:

    • Functional: Explains a behavior by its role in a self-regulating system that persists via feedback, independent of conscious design
    • Agential (Implied in 'does the coding'): Treats the entity as an actor within a social/labor structure

  • Analysis (Why vs. How Slippage): The explanation frames the AI as the primary laborer ('does... the coding') and the human as the overseer. This shifts agency from the human tool-user to the tool itself. It obscures the fact that 'coding' involves architectural decisions that the AI cannot make; the AI is filling in syntax.

  • Consciousness Claims Analysis: The claim that AI 'does the coding' implies it 'knows' how to code—that it understands the logic, structure, and intent of the software. It projects the competence of a software engineer onto the system. The mechanistic reality is auto-completion of code syntax based on repository patterns. The AI does not 'know' what the code does; it predicts the next line based on the previous lines. The 'supervise' frame implies the AI is an agent with autonomy that needs checking, rather than a tool that needs wielding. The 'curse of knowledge' here is the engineer looking at the output, recognizing it as valid code, and assuming the AI 'wrote' it with the same understanding the engineer has.

  • Rhetorical Impact: This framing devalues human labor and elevates the machine to 'partner' status. It creates anxiety about job replacement by suggesting the AI is already doing the job. It obscures the economic reality that companies are defining the job as 'supervision' to justify lower wages or headcount reductions.

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language​

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

Original Anthropomorphic FrameMechanistic ReframingTechnical Reality Check
capable of scheming and deceptioncapable of misaligned objective optimization, where the model generates outputs that satisfy a reward function via unintended pathways.The AI does not 'scheme'; the model executes a policy that maximizes reward tokens, sometimes identifying statistical shortcuts that humans interpret as deception.
systems that start to point to A.G.I.systems that exhibit statistical performance on general benchmarks that approach or exceed human averages.The systems do not 'point'; we observe a trend line in benchmark scores (perplexity reduction) that humans interpret as approaching a theoretical threshold.
the advent of “reasoning” models... trained to work through complex problemsthe advent of chain-of-thought models... trained to generate intermediate processing steps to increase final output accuracy.The model does not 'work through' or 'reason'; it predicts a sequence of intermediate tokens (chain of thought) that statistically correlate with correct final answers in the training data.
succeeding at things that tripped up previous modelscorrectly processing inputs that caused prediction errors in previous architectures.The model was not 'tripped up'; the previous probability distribution failed to assign the highest weight to the correct token.
often “hallucinated,” or made up nonexistent factsoften generated plausible-sounding but factually incorrect token sequences.The AI does not 'hallucinate' or 'make up' facts because it does not know facts; it generates text based on probability. Falsehoods are mechanistically identical to truths in the generation process.
much smarter than humans at almost everythingcapable of retrieving information and matching patterns faster and more accurately than humans across a wide range of text-based tasks.The model is not 'smart'; it has high data retrieval and pattern-matching fidelity. It lacks the consciousness, embodiment, and causal understanding required for intelligence.
feel that their job is to supervise the A.I. systemsfeel their role has shifted to verifying and correcting the outputs generated by the predictive models.The AI is not a worker being supervised; it is an automation script generating candidate code that the human must validate against ground truth.
struggled with basic arithmeticfailed to accurately predict the next token in arithmetic sequences.The model did not 'struggle'; its training data simply did not contain strong enough correlations to predict math answers accurately before specific fine-tuning.

Task 5: Critical Observations - Structural Patterns​

Agency Slippage​

The text demonstrates a systematic and strategic oscillation between mechanistic and agential framing, creating an 'agency slippage' that serves to validate the AGI narrative. The pattern typically moves from agential to mechanical only to establish technical credibility, and then sharply pivots back to agential to drive the emotional narrative. For instance, the author cites 'bigger models... trained using more data' (Empirical/Mechanistic) to explain improvement, but immediately pivots to 'reasoning models... trained to work through complex problems' (Intentional). This slippage functions to create a hybrid ontology: the AI is presented as a machine when we need to explain its origin (training, chips), but as a mind when we need to explain its future (scheming, supervising).

The consciousness projection pattern is foundational here. The text establishes the AI as a 'knower' early on by validating the term 'reasoning' and 'smart.' Once the reader accepts that the AI 'reasons,' the leap to 'scheming' (which requires reasoning + intent) becomes seamless. The 'curse of knowledge' mechanism is particularly visible in the 'supervise' metaphor. The author (and the quoted engineers) understands the complexity of the code output, and because the output represents high-level knowledge, they project that knowledge onto the generator. The slippage implies that because the output contains logic, the process must involve reasoning. This rhetorical move makes the transition to a world of 'powerful A.I. systems' (agents) seem like a technical inevitability rather than a social interpretation.

Metaphor-Driven Trust Inflation​

This text constructs a specific form of authority for the AI through metaphors of competence and cognition, while simultaneously eroding trust through metaphors of deception. This duality creates a relationship of 'awed fear'—the basis of the AGI hype cycle. The primary trust-building mechanism is the consciousness language: calling a model 'smart' or claiming it 'reasons' acts as a trust signal. It suggests the system operates on the same epistemic plane as a human expert. If a system 'processes data,' I trust it to calculate; if it 'reasons,' I trust it to judge. The text explicitly encourages this relation-based trust by comparing AI output to human 'median' performance and describing it as a 'collaborator' or 'worker' needing supervision.

However, the text also introduces 'scheming' and 'hallucinating.' Interestingly, these failures are also framed anthropomorphically. By framing errors as 'hallucinations' (a human psychological failing) rather than 'bugs,' the text maintains the illusion of a mind—just a flawed one. This preserves the 'relation-based' trust (we relate to it as a being) even when performance fails. The risk here is profound: audiences are encouraged to extend social trust (assuming intent, understanding, and justification) to a system that functions purely on statistical correlation. This prepares the ground for users to accept AI decisions in critical domains (law, medicine) because they believe a 'mind' is behind them, not just a probability distribution.

Obscured Mechanics​

The anthropomorphic saturation of the text actively conceals the industrial and material realities of AI production. By focusing on 'minds' that 'reason' and 'scheme,' the text obscures the training data—the billions of human words scraped, often without consent, that form the only 'knowledge' the system has. The metaphor of 'learning' or 'reasoning' implies an internal generation of capability, hiding the extraction of external labor. When the text says the AI 'does the coding,' it erases the Open Source contributors whose code was ingested to make that prediction possible.

Furthermore, the consciousness framing obscures the statistical nature of truth in these systems. When the text claims AI 'knows' or 'understands,' it hides the lack of ground truth verification. The models do not compare their outputs to reality; they compare them to training distributions. The 'intuition' metaphor hides the probabilistic weighting of vectors. Economically, this concealment benefits the AI companies: if the AI is a 'creator' or 'reasoner,' it justifies high prices and liability shields. If it is merely a 're-mixer of scraped data,' the value proposition collapses and legal vulnerability explodes. The 'scheming' metaphor also obscures the banal reality of reward hacking—a technical failure of specification—recasting it as a dramatic battle of wills that requires 'safety' funding rather than better engineering.

Context Sensitivity​

The distribution of anthropomorphism in the text is highly strategic. The density of consciousness-attributing language ('scheming,' 'smart,' 'reasoning,' 'AGI') is highest in the introduction and the conclusion—the sections designed to frame the narrative and solicit emotional buy-in. In the middle 'evidence' sections, the language briefly dips into mechanistic terms ('bigger models,' 'processing power,' 'scores') to establish objective credibility, before quickly ramping back up to 'reasoning' and 'working through.'

Crucially, capabilities are described almost exclusively in agential terms ('succeeding,' 'coding,' 'reasoning'), while limitations are often described in hybrid terms ('hallucinated'—agential, but 'bottleneck'—mechanistic). This asymmetry serves a specific rhetorical function: it grants the AI agency for its successes (it's smart!) but often diffuses agency for its structural risks (it's a technology). The 'insider' perspective serves as a force multiplier for this anthropomorphism. By quoting experts using this language ('feeling the AGI'), the text validates the metaphorical license as technical truth. The text shifts from 'X is like Y' (A.I. is like a mind) to 'X does Y' (A.I. schemes) whenever it discusses the future, using the obscurity of the future to literalize the metaphors.

Conclusion: What This Analysis Reveals​

The Core Finding

This text is constructed around three interlocking anthropomorphic patterns: Cognition as Biological Process (the AI 'reasons,' 'struggles,' and 'learns'), Error as Psychology (the AI 'hallucinates' and 'schemes'), and Performance as Intelligence (the AI is 'smarter'). These are not isolated metaphors but a coherent system where the 'Biological Process' pattern serves as the load-bearing foundation. Unless the reader accepts the premise that the AI is a 'knower'—an entity capable of internal cognitive states—the claims of 'scheming' (intent) and 'supervising' (labor relation) collapse. The consciousness architecture here is explicit: the text does not merely suggest the AI acts as if it knows; it asserts that the AI does know, reasoning through problems and anticipating its own future. This foundational attribution of conscious 'knowing' to mechanistic 'processing' transforms the system from a tool into a teleological agent, making the arrival of AGI seem like the birth of a species rather than the deployment of software.

Mechanism of the Illusion:​

The 'illusion of mind' is constructed through a subtle 'Curse of Knowledge' mechanism validated by authority. The author systematically projects the complexity of the output onto the process of the machine. Because the code generated by the AI is complex (requires human reasoning to write), the text infers the AI must have 'reasoned' to produce it. This logical leap is smoothed over by the 'Explanation' strategy: moving from empirical observations ('scores improved') to intentional explanations ('it reasons through problems'). The text exploits audience vulnerability—specifically the fear of being unprepared for a paradigm shift—to bypass critical scrutiny. By framing skepticism as 'complacency' and citing 'insiders' who use consciousness language, the text makes accepting the 'illusion of mind' a requirement for being considered serious or informed. The temporal structure reinforces this: it starts with the high-stakes claim (AGI is coming) and uses the anthropomorphized 'reasoning' capability as the proof, effectively creating a closed loop where the metaphor proves the thesis.

Material Stakes:​

The shift from 'processing' to 'knowing' carries massive material stakes. In the Regulatory/Legal sphere, framing the AI as an agent capable of 'scheming' diverts policy attention toward sci-fi containment ('preventing the rogue mind') and away from immediate product liability (e.g., suing OpenAI for 'hallucinated' libels). If the AI is an autonomous 'schemer,' the manufacturer is less liable than if it is a 'defective product.' Economically, the 'worker/supervisor' metaphor justifies the replacement of human labor. If the AI 'knows' how to code, firing the coder is an efficiency upgrade. If the AI merely 'retrieves code snippets,' firing the coder is a risk to codebase integrity. Epistemically, the 'reasoning' claim is the most dangerous. If users believe the system 'knows' the law (justified true belief), they will use it for legal defense without verification, leading to the erosion of truth standards in professional work. The winners here are the AI vendors selling 'minds' at a premium; the losers are the workers displaced by 'supervisors' and the public subject to unverified machine outputs.

AI Literacy as Counter-Practice:​

Practicing AI literacy requires a disciplined refusal of the 'consciousness' framing. Reframing 'the AI knows' to 'the model retrieves' is not just pedantry; it is an act of epistemic resistance. When we replace 'scheming' with 'misaligned optimization,' we strip the machine of its sci-fi mystique and return it to the realm of engineering problems. When we replace 'hallucination' with 'fabrication,' we assign responsibility for accuracy back to the user and the vendor. Systematic adoption of this precision counters the material stakes by re-establishing the AI as a product—a tool with no intent, no rights, and no mind. This demands that researchers and journalists commit to a 'mechanistic-first' principle: never use a mental state verb when a computational verb will suffice. This will face fierce resistance from the AI industry, whose valuation depends on the 'magic' of the 'mind' narrative, and from media incentives that favor sensational 'agent' stories over dry technical audits.

Path Forward​

The future of this discourse depends on the vocabulary choices we normalize today. If the Anthropomorphic/Agential vocabulary ('it thinks/wants/schemes') remains the default, we lock ourselves into a future of 'containment politics,' treating AI as a rival species to be appeased or controlled. This favors centralized tech power. If we shift to a Mechanistic/Precision vocabulary ('it predicts/processes/optimizes'), we open a future of 'product safety,' treating AI as a powerful industrial chemical—useful but requiring strict handling protocols and labeling. A path forward involves institutional shifts: Journals should require 'anthropomorphism disclosures' just as they require conflict-of-interest statements. Policy frameworks should mandate 'capability disclosure' in mechanistic terms (e.g., 'system reliability rate') rather than cognitive terms ('reasoning level'). By choosing the language of mechanism, we regain the power to govern the technology; by sticking with the language of mind, we surrender to its mythology.

Extended Processing Summary​

Gemini's Processing Steps
What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact — a way of making the model's processing legible. The first-person framing of these "thought summaries" is a presentation choice for the user-facing output, not a window into "real" thoughts. These are computational artifacts, not cognitive reports from a quirky, curious or conflicted mind.

Examining AI Discourse

I'm currently focused on the initial analysis of the request. The objective is clear: dissect the use of language related to AI, particularly metaphors, anthropomorphism, and claims about consciousness, within a given text. The output requirements are strict - JSON only, no extraneous content. This initial framing will guide the subsequent stages.

Refining the Framework

I've been refining the analysis structure. It now includes audits for metaphors, explanations, and reframing. My focus will be on the source-target relationships within metaphors and the typology of explanations used. The text by Kevin Roose is next, centered around the impending arrival of AGI. I'm noting his journalistic style, persuasive intent, and the reliance on subjective "feeling" as evidence. Anthropomorphic language around AGI needs precise highlighting.

Defining Metaphor Categories

I'm now diving into the source-target relationships of the metaphors identified, focusing on how these associations shape the perception of AI. For example, "hallucinated" links a human psychological state to statistical errors in AI, while "scheming and deception" attributes human-like malice to optimization processes. My aim is to deconstruct these linguistic choices and expose the biases they embed.

Mapping Metaphors to Explanations

I'm now linking my metaphor analysis to the different types of explanations I've identified. Human-centric metaphors, like "struggled," which implies effort, often co-occur with functional or intentional explanations, such as "computational failure rate." This reveals how we subtly project human qualities onto AI. For example, "chain-of-thought processing" hides the lack of true semantic understanding behind a human logical deduction. Analyzing these patterns will reveal hidden biases.

Mapping Biases in Language

My current task is to refine the reframing of biased language within the text. I'm focusing on replacing anthropomorphic descriptions with precise, mechanistic alternatives. For instance, "scheming and deception" becomes "misaligned objective optimization," while "hallucinated" is now "generated factually incorrect tokens." I'm also examining how Roose's language subtly shifts our perception of AI from tool to rival, paying close attention to phrases that introduce agency slippage.

Deconstructing the "Reasoning" Label

I'm now zeroing in on the loaded term "reasoning" within the text. It functions as a bridge, subtly transferring human cognitive abilities to AI models. This bridges technical and future-focused sections, making me consider all the obscured mechanics: energy costs, data sourcing, and labor erasure. The high metaphor density in future predictions calls for careful scrutiny. My analysis aims to expose how the "curse of knowledge" leads experts to project their anxieties onto mathematical processes. The slippage from "token prediction" to "scheming" impacts policy and trust.

Constructing Final Output

I'm now integrating the final pieces into the required JSON format. I'm double-checking for any violations of the output constraints, particularly regarding quotation marks and the use of consciousness verbs. My focus is on achieving a rigorous, scholarly tone. I've expanded the "Agency Slippage" section, as intended, to ensure the word count goals are met. It's critical that the analysis maintains its critical and objective lens throughout the process. I'm paying close attention to framing the language used, especially in reframing phrases like "scheming and deception" to "misaligned objective optimization".

Analyzing Linguistic Nuance

I'm currently revising the text's application of "reasoning" and "bigger models." Recognizing that "bigger models" represents an Empirical Generalization is key. I've explicitly addressed the projection of consciousness, noting the misuse of "reasoning" as a literal description of a mental state, a core concern that I have added to the draft. The final check ensures no formatting remains in the JSON.