Skip to main content

AI & Human Co-Improvement for Safer Co-Superintelligence

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping), the philosophy of social science (Robert Brown's typology of explanation), and accountability analysis.

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.

Task 1: Metaphor and Anthropomorphism Audit​

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

V3 Enhancement: Each metaphor now includes an accountability analysis.

1. The AI as Collegial Partner​

Quote: "Our central position is that 'Solving AI' is accelerated by building AI that collaborates with humans to solve AI."

  • Frame: Model as Professional Colleague
  • Projection: This metaphor projects complex social agency, shared intentionality, and mutual understanding onto the software. By using 'collaborates,' the text implies the AI possesses a theory of mind—the ability to understand a shared goal, recognize the human's contribution, and intentionally coordinate its actions to assist. It suggests a symmetrical relationship of two minds working together, rather than a human using a tool. This elevates the system from a probabilistic text generator to a social agent capable of professional partnership.
  • Acknowledgment: Presented as direct description.
  • Implications: Framing the system as a 'collaborator' creates an 'illusion of mind' that inflates trust. If users believe they are collaborating with an entity that 'understands' the shared goal, they may overestimate the system's ability to fact-check, reason, or adhere to ethical norms. This anthropomorphism risks inducing users to defer to the system's 'judgment' as they would a human peer, obscuring the fact that the 'collaboration' is merely the system completing patterns based on statistical likelihoods without any concept of the research goal itself.

Accountability Analysis:

  • This framing displaces the agency of the system's designers (Meta/FAIR researchers). An AI does not 'collaborate'; humans design interfaces and objective functions that reward specific output patterns. By framing the interaction as 'collaboration,' the text obscures the power dynamic: the human user is training or utilizing a product owned by a corporation. It suggests a voluntary partnership, hiding the fact that the 'collaborator' is a tool designed to extract data and labor from the human 'partner' to improve its own metrics (as admitted in the 'Co-improvement' definition).
Show more...

2. Cognition as a Discrete Puzzle​

Quote: "Solving AI"

  • Frame: Intelligence as Math Problem
  • Projection: This metaphor reifies 'AI' (intelligence/consciousness) as a discrete, bounded puzzle or equation that can be 'solved.' It projects a teleological endpoint onto the development of information processing systems, suggesting that intelligence is a destination or a state that can be achieved once and for all. It implies that 'intelligence' is a technical hurdle to be cleared rather than an open-ended, context-dependent social and biological capacity.
  • Acknowledgment: Scare quotes are used initially ('Solving AI') but the concept is treated as a concrete operational goal throughout.
  • Implications: This framing implies that creating superintelligence is a technical inevitability and a valid engineering objective. It strips 'intelligence' of its embodied, social, and ethical dimensions, reducing it to a metric. This encourages a 'race' dynamic where the only goal is to 'solve' the problem first, potentially justifying reckless deployment or safety shortcuts under the guise of scientific imperative. It obscures the risk that 'solving' AI might actually mean 'automating critical human functions without oversight.'

Accountability Analysis:

  • Who decided that AI needs to be 'solved'? This framing naturalizes the commercial goals of tech companies as scientific imperatives. It obscures the specific human actors (executives at Meta, OpenAI, Google) who have defined 'Solving AI' as the maximization of benchmark scores. It frames the enterprise as a universal quest for humanity ('positive solution for humanity') rather than a corporate product roadmap, diffusing the responsibility for the societal disruption caused by this 'solution.'

3. Recursive Agency​

Quote: "models that create their own training data, challenge themselves to be better"

  • Frame: Model as Autodidact / Aspiring Student
  • Projection: This maps the human qualities of aspiration, self-reflection, and intentional self-improvement onto the system. 'Challenge themselves' implies the model has a self-concept, a desire to improve, and the agency to set challenges. It suggests a conscious internal loop where the system 'wants' to get better, rather than a mechanical optimization process driven by loss functions designed by humans.
  • Acknowledgment: Presented as direct description.
  • Implications: This is a profound consciousness projection. It suggests the AI is an agent with its own internal drive. This inflates the perceived autonomy of the system, leading to fears of 'runaway' self-improvement (the 'Paperclip Maximizer' scenario) or unwarranted trust in the system's 'dedication.' Mechanistically, the model creates data because code executes a generation script; it 'challenges' itself because a loop feeds output back as input. Attributing this to the model's 'self' mystifies the engineering process.

Accountability Analysis:

  • This construction completely erases the engineers. 'Models create their own data' hides the fact that engineers chose to implement synthetic data generation pipelines to bypass data scarcity. 'Challenge themselves' hides the specific reward functions and prompts written by researchers to force this behavior. It attributes the 'desire' for improvement to the software, protecting the developers from scrutiny regarding the decision to build recursively self-amplifying systems.

4. Ecological Mutualism​

Quote: "endow both AIs and humans with safer superintelligence through their symbiosis"

  • Frame: Software as Biological Symbiont
  • Projection: This metaphor maps biological interdependence onto the human-machine relationship. 'Symbiosis' implies a natural, organic, and mutually beneficial life-cycle integration. It suggests the AI is a living organism that 'lives' with the human, and that this union is a natural step in evolution rather than a product deployment strategy.
  • Acknowledgment: Presented as direct description.
  • Implications: Symbiosis implies necessity—that humans need the AI to survive or thrive, and vice versa. This naturalizes the deep integration of corporate surveillance and automation technologies into human life. It frames dependency on AI as 'evolution' rather than 'addiction' or 'vendor lock-in.' It creates a false sense of security (symbionts generally don't destroy their hosts) that obscures the predatory economic nature of data extraction.

Accountability Analysis:

  • Who benefits from the 'symbiosis' framing? Meta and other AI vendors. It reframes 'user dependency on our platform' as 'biological destiny.' The 'actor' here is the corporation seeking to make its product indispensable. By calling it 'symbiosis,' the text obscures the power asymmetry: the human user generates value (data, feedback) that the corporation captures. The 'organism' the human is symbiotic with is not the code, but the corporate entity itself.

5. Teleological Inevitability​

Quote: "we are marching towards ever more intelligent AI systems"

  • Frame: Development as Military March / Destiny
  • Projection: This maps AI development onto a physical, collective, forward movement (a 'march'). It implies a unified vector of progress, inevitability, and a destination. It suggests that 'we' (humanity? researchers?) are all moving in this direction together and that the increase in intelligence is a natural law like gravity.
  • Acknowledgment: Presented as direct description.
  • Implications: This framing removes the element of choice. It presents 'superintelligence' as something that is coming regardless of human decision, rather than something being built by specific companies. This induces passivity in policymakers and the public—if we are 'marching towards' it, we can't stop it, only 'steer' it. It obscures the possibility of a moratorium or a different developmental path.

Accountability Analysis:

  • Who is 'marching'? The text says 'we,' implicating the reader and humanity in a corporate roadmap. In reality, a small group of tech executives and researchers are driving this development. The passive framing ('marching towards') hides the active decisions to scale models, buy GPUs, and deploy unproven systems. It diffuses responsibility for the consequences of this 'march' onto the 'field' or 'history' rather than the specific individuals pushing the pace.

6. The Cosmic Eclipse​

Quote: "before AI eclipses humans in all endeavors"

  • Frame: Obsolescence as Celestial Event
  • Projection: This metaphor maps the replacement of human labor and capability onto a celestial event (an eclipse). It suggests a massive, natural, unavoidable phenomenon where one body naturally overshadows another. It implies scale, dominance, and the natural order of things.
  • Acknowledgment: Presented as direct description.
  • Implications: This is a fatalistic metaphor that creates a sense of helplessness. An eclipse cannot be stopped; it can only be endured. This prepares the audience to accept human obsolescence as a natural cosmic event rather than a socio-economic choice made by those deploying automation. It shifts the focus from 'protecting human roles' to 'surviving the eclipse.'

Accountability Analysis:

  • This is the ultimate accountability sink. An eclipse has no author. By framing labor displacement as an 'eclipse,' the authors erase the employers and corporations making the decision to replace human workers with software. It obscures the economic incentives driving this replacement and frames it as a capability threshold ('when AI is smarter') rather than a profitability threshold ('when AI is cheaper').

7. The Research Agent​

Quote: "autonomous AI research agents... conducting research with humans"

  • Frame: Software as Occupational Role
  • Projection: This projects the social role, professional judgment, and institutional identity of a 'researcher' onto a software program. It implies the system follows the scientific method, understands hypotheses, and adheres to academic norms, rather than just pattern-matching literature and generating plausible-sounding text.
  • Acknowledgment: Presented as direct description ('research agents').
  • Implications: This threatens the epistemic integrity of science. If software is treated as a 'researcher,' its hallucinations may be treated as 'findings.' It conflates 'generating text about science' with 'doing science.' It risks polluting the scientific record with non-reproducible, statistically generated noise disguised as research, because the 'agent' metaphor implies a level of verification and intent that doesn't exist.

Accountability Analysis:

  • Calling the software a 'research agent' allows the human authors to offload the labor of verification. If the 'agent' makes a mistake, it's a 'glitch' in the collaborator. This serves the interest of high-volume publication. It also obscures the specific human researchers who are choosing to automate their own field. The 'actor' is the human who decides to treat an unchecked output as a valid scientific contribution.

8. The Goal Pathologizer​

Quote: "suffer from goal misspecification"

  • Frame: Design Flaw as Medical Condition
  • Projection: By saying the system 'suffers,' the text attributes a capacity for experiencing negative states to the software. More broadly, 'goal misspecification' implies the system has a goal that it is trying to achieve, and the problem is just that the goal was specified wrong. It treats the system as a goal-seeking agent rather than a function minimizer.
  • Acknowledgment: Technical term treated as description.
  • Implications: This obscures the mechanical reality that the system has no goals, only mathematical loss landscapes. It implies the AI is 'trying' to do the right thing but is 'confused.' This builds sympathy and trust. It also suggests the solution is just 'better specification' (technical fix) rather than questioning whether a system that blindly optimizes a metric should be deployed at all.

Accountability Analysis:

  • Who specified the goal? The engineers. Who decided to deploy a system where 'misspecification' leads to harm? The executives. The passive construction 'suffer from goal misspecification' hides the 'specifier.' It frames the danger as an inherent property of the complexity of AI (a disease it catches) rather than a direct result of the developers' inability to write safe code.

Task 2: Source-Target Mapping​

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: Human Professional Collaboration → Human-Computer Interaction (Prompting/Feedback Loops)​

Quote: "building AI that collaborates with humans to solve AI"

  • Source Domain: Human Professional Collaboration
  • Target Domain: Human-Computer Interaction (Prompting/Feedback Loops)
  • Mapping: The structure of human collaboration (shared mental states, mutual intent, division of labor based on expertise, social contract) is mapped onto the interaction between a user and a language model. It implies the model 'intends' to help, 'understands' the research context, and 'contributes' novel ideas.
  • What Is Concealed: This conceals the mechanical reality: the user provides input (prompts), and the model generates output based on statistical correlations in its training data. There is no 'shared goal' in the machine; there is only a forward pass through a neural network. It hides the lack of consent, the lack of understanding, and the fact that the 'collaboration' is completely one-sided (the human directs, the machine computes).
Show more...

Mapping 2: Autodidactic Student / Organic Growth → Recursive Synthetic Data Generation & Optimization​

Quote: "models that create their own training data... challenge themselves to be better"

  • Source Domain: Autodidactic Student / Organic Growth
  • Target Domain: Recursive Synthetic Data Generation & Optimization
  • Mapping: The structure of a student learning (self-reflection, identifying weaknesses, creating study plans, internal drive) is mapped onto automated scripts where a model's output is filtered and fed back as input for the next training round. It implies an internal locus of control and a desire for improvement.
  • What Is Concealed: It conceals the 'human in the loop' who wrote the script, set the threshold for 'better,' and initiated the process. It hides the mechanical circularity: the model is not 'challenging itself'; it is collapsing into its own distribution unless externally guided. It obscures the risk of 'model collapse' (degeneration of quality) by framing it as 'improvement.'

Mapping 3: Biological Symbiosis → Software Integration / Human-Computer Dependency​

Quote: "endow both AIs and humans with safer superintelligence through their symbiosis"

  • Source Domain: Biological Symbiosis
  • Target Domain: Software Integration / Human-Computer Dependency
  • Mapping: Biological relationships (mutualism, survival dependence) are mapped onto software usage. It implies the relationship is natural, necessary for survival, and mutually life-sustaining. It suggests the AI is a living entity that evolves alongside the human.
  • What Is Concealed: It conceals the commercial nature of the relationship (Vendor-Customer). Symbiosis implies an inescapable biological bond; software is a product that can be uninstalled. It hides the power dynamics: the 'symbiont' is owned by a third party (Meta) and extracts data from the host. It mystifies the code as a life form.

Mapping 4: Human Researcher / Scientist → Automated Literature Review & Text Generation Scripts​

Quote: "autonomous AI research agents"

  • Source Domain: Human Researcher / Scientist
  • Target Domain: Automated Literature Review & Text Generation Scripts
  • Mapping: The role of a scientist (hypothesizing, experimenting, deducing, publishing) is mapped onto a script that retrieves papers, summarizes them, and generates new text following the format of a paper. It implies the output contains 'knowledge' or 'discovery.'
  • What Is Concealed: It conceals the lack of ground truth. A model cannot 'experiment' in the physical world (usually); it simulates or hallucinates results based on text patterns. It hides the distinction between 'scientific sounding text' and 'science.' It obscures the absence of critical thinking and accountability—if the 'agent' fabricates data, it has no professional reputation to lose.

Mapping 5: Mathematical Problem / Puzzle → Developing General Purpose Computing Systems​

Quote: "Solving AI"

  • Source Domain: Mathematical Problem / Puzzle
  • Target Domain: Developing General Purpose Computing Systems
  • Mapping: The structure of a puzzle (a defined initial state, a clear goal state, a solution path) is mapped onto the open-ended development of cognitive technologies. It implies there is a correct 'answer' or 'final state' for AI.
  • What Is Concealed: It conceals the fact that 'intelligence' is not a single problem but a contestable concept. It hides the social and political choices involved in defining what 'solved' looks like (e.g., solved for whom? The CEO or the worker?). It obscures the open-ended, continuous nature of technology maintenance and the impossibility of a 'final' solution.

Mapping 6: Celestial Mechanics (Eclipse) → Labor Market Displacement / Capability Thresholds​

Quote: "before AI eclipses humans"

  • Source Domain: Celestial Mechanics (Eclipse)
  • Target Domain: Labor Market Displacement / Capability Thresholds
  • Mapping: The irresistible, scale-invariant movement of celestial bodies is mapped onto the development of software capabilities. It implies the process is governed by natural laws, is predictable, and is unstoppable by human agency.
  • What Is Concealed: It conceals the economic decisions. Humans are not 'eclipsed' by AI; they are fired by managers who replace them with AI. It hides the specific benchmarks being used to claim superiority. It mystifies the technology, treating it as a force of nature rather than a collection of engineering choices.

Mapping 7: Pathology / Victimhood → Engineering Error / Objective Function misalignment​

Quote: "suffer from goal misspecification"

  • Source Domain: Pathology / Victimhood
  • Target Domain: Engineering Error / Objective Function misalignment
  • Mapping: The state of a patient suffering from a condition is mapped onto a software system executing a poorly written objective function. It implies the system is a victim of its own code and has an 'internal' state of health that is compromised.
  • What Is Concealed: It conceals the agency of the programmer. The model does not 'suffer'; it executes. It hides the fact that the 'misspecification' is actually the system doing exactly what it was told to do, which happened to be harmful. It externalizes the error as a 'condition' rather than a 'mistake.'

Mapping 8: Human Cognitive Awareness → Lack of Specific Training Token Associations​

Quote: "models do not 'understand' they are jailbroken"

  • Source Domain: Human Cognitive Awareness
  • Target Domain: Lack of Specific Training Token Associations
  • Mapping: This is a negative mapping, but it uses the source domain of 'understanding' to describe a mechanical gap. By saying they don't understand this specific thing, it implies the category of 'understanding' is applicable to them in principle.
  • What Is Concealed: It conceals the fact that they don't 'understand' anything in the human sense. By specifying they don't understand jailbreaking, it leaves open the possibility that they do understand other things (like the 'collaboration' mentioned elsewhere). It treats the lack of a meta-cognitive state as a specific deficit rather than a fundamental property of the system.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")​

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1​

Quote: "Our central position is that 'Solving AI' is accelerated by building AI that collaborates with humans to solve AI... Instead, we advocate for co-improvement, whereby collaborative AI agents are built with the goal of conducting research with humans."

  • Explanation Types:

    • Intentional: Refers to goals or purposes and presupposes deliberate design, used when the purpose of an act is puzzling.
    • Teleological / Functional: Explains a behavior by its role in a self-regulating system (the research loop) or its directedness toward an end (Solving AI).

  • Analysis (Why vs. How Slippage): This passage frames the AI primarily through an Intentional lens. The AI is built 'with the goal of conducting research,' and it 'collaborates.' This slips between the designers' goal (to build a tool) and the system's goal (to conduct research). It emphasizes the 'why' (purpose: solving AI) over the 'how' (mechanism: processing data). This choice obscures the mechanical reality that the AI has no goals; it effectively transfers the designers' intent into the object, animating it.

  • Consciousness Claims Analysis: The text uses high-stakes consciousness language: 'collaborates,' 'conducting research,' 'solving.' These imply a shared epistemic state. To 'collaborate' requires knowing what the partner knows and what the goal is. To 'solve' implies understanding the problem.

Consciousness Projection: The text treats the AI as a 'Knower'—an entity capable of holding the goal of 'research' in its mind. Mechanistically, the AI is a 'Processor' that minimizes prediction error.

Curse of Knowledge: The authors (experts) know what 'solving AI' means to them. They project this comprehension onto the model, assuming that when the model outputs text about AI research, the model 'knows' it is doing research.

Concealed Distinction: The actual process is: The model generates tokens that are statistically likely to follow prompts about AI research. It does not 'conduct research'; it simulates the language of research.

  • Rhetorical Impact: This framing constructs the AI as a competent partner. It creates an expectation of autonomy and reliability. If the audience believes the AI is 'collaborating' to 'solve AI,' they will trust its outputs as intellectual contributions. This diffuses the risk perception—users feel they are working with a smart colleague, not using a probabilistic tool. It legitimizes the output as 'research' rather than 'generated text,' validating the automation of scientific labor.
Show more...

Explanation 2​

Quote: "models that create their own training data, challenge themselves to be better, and learn to evaluate and reward themselves"

  • Explanation Types:

    • Genetic: Traces origin or development through a dated sequence of events or stages.
    • Intentional: Refers to goals or purposes and presupposes deliberate design.

  • Analysis (Why vs. How Slippage): This explanation hybridizes a Genetic account (how models evolved over time) with intense Intentional language ('challenge themselves,' 'reward themselves'). It frames the mechanism of recursive training (a script feeding output back as input) as an act of will or self-improvement. This emphasizes agency and autonomy, obscuring the deterministic nature of the code execution.

  • Consciousness Claims Analysis: The verbs 'challenge,' 'evaluate,' and 'reward' (reflexive) suggest a conscious self-concept. A system cannot 'challenge itself' without a concept of 'self' and 'difficulty.'

Knowing vs Processing: The text implies the AI 'knows' its own limitations and seeks to overcome them. Mechanistically, a loss function calculates a value, and gradients are updated. There is no 'self' to be challenged.

Curse of Knowledge: The authors know they designed the system to improve. They project this intent onto the system, describing the mechanical optimization as the system's own 'challenge.'

Concealed Distinction: 'Challenge themselves' → 'The system's outputs are scored by a discriminator, and weights are adjusted to maximize that score.'

  • Rhetorical Impact: This creates the 'Self-Improving AI' mythos—the idea that the machine has a will to power. It generates both hype (unlimited capability) and fear (loss of control). It positions the AI as an independent actor in the world, distinct from its creators, which helps shield the creators from liability for what the 'autonomous' system does.

Explanation 3​

Quote: "models do not 'understand' they are jailbroken"

  • Explanation Types:

    • Theoretical: Embeds behavior in a deductive or model-based framework (mental state attribution/denial).

  • Analysis (Why vs. How Slippage): This is a fascinating negative explanation. It explains the failure (jailbreaking) by the absence of a mental state ('understanding'). Even in denial, it frames the AI's operation in psychological terms rather than mechanical ones (e.g., 'the model lacks training examples for this adversarial pattern'). It emphasizes the cognitive deficit rather than the structural vulnerability.

  • Consciousness Claims Analysis: Consciousness Verbs: Uses 'understand' (negated).

Projection: This is a strategic retreat from consciousness projection. When discussing failure/risk, the authors retreat to 'it doesn't understand.' This reveals the malleability of the framing: the AI 'collaborates' (high consciousness) when it works, but 'doesn't understand' (low consciousness) when it fails.

Curse of Knowledge: The authors know what jailbreaking is. They attribute the failure to the model 'not knowing' this concept, rather than the model simply following its probability distribution which happens to include the jailbreak response.

Concealed Distinction: 'Does not understand it is jailbroken' → 'The model has no meta-cognition or context-window state that tracks violation of safety guidelines; it simply predicts the next token based on the adversarial prompt.'

  • Rhetorical Impact: This preserves the 'magic' of the system while excusing its failures. By saying it 'doesn't understand,' it implies that if we just gave it more capability (made it understand), the safety problem would be solved. It frames safety as a capabilities problem (needs more knowing) rather than a control problem. It maintains the anthropomorphic frame even in failure.

Explanation 4​

Quote: "AI augments and enables humans in all areas of society, rather than pursuing full automation that removes human decision-making."

  • Explanation Types:

    • Intentional: Refers to goals or purposes and presupposes deliberate design.

  • Analysis (Why vs. How Slippage): This attributes the high-level socio-economic goal ('augments... rather than pursuing') to the 'AI' (or the 'solution' involving AI). It creates an ambiguity: is it the AI that pursues this, or the researchers? The grammar allows the AI to be the agent of benevolence ('AI augments'). It emphasizes the helpful 'why' to distract from the displacement 'how.'

  • Consciousness Claims Analysis: Consciousness Verbs: 'Augments,' 'enables' (agential verbs).

Projection: It attributes a benevolent social role to the technology.

Curse of Knowledge: The authors have a vision of 'Co-improvement.' They project this vision onto the tool, speaking as if the tool itself shares the goal of augmentation.

Concealed Distinction: The AI does not 'pursue' anything. It generates outputs. Whether those outputs 'augment' or 'automate' depends entirely on the employer's decision of how to use them. The text conceals the economic decision-maker (the employer) behind the 'behavior' of the AI.

  • Rhetorical Impact: This is a 'Trust' framing. It reassures the audience that the AI is 'on our side.' It obscures the labor reality: that 'augmentation' often is a euphemism for 'training the replacement' or 'de-skilling the worker.' By attributing this benevolent orientation to the AI/paradigm, it hides the corporate interests that might prefer full automation if it were cheaper.

Explanation 5​

Quote: "with the help of AI we are more likely to solve the capability and safety problems of AI — but with humans in the loop, collaborating on the research."

  • Explanation Types:

    • Functional: Explains a behavior by its role in a self-regulating system.
    • Methodological / Reason-Based: Gives the rationale for acting (humans in loop = safer).

  • Analysis (Why vs. How Slippage): This explains the method (human-in-the-loop) via its function (safety/speed). It frames the AI as a tool ('with the help of') but immediately elevates it to a partner ('collaborating'). It emphasizes the synergy of the two components. It blurs the line between 'using a tool' and 'working with a partner.'

  • Consciousness Claims Analysis: Consciousness Verbs: 'Collaborating,' 'solve.'

Projection: It implies the AI contributes something distinct from the human—a separate intelligence that 'helps.'

Curse of Knowledge: The authors know the AI provides faster processing. They conflate 'faster processing' with 'collaborative problem solving.'

Concealed Distinction: 'Collaborating on the research' → 'Using the model to generate code, summaries, or ideas that the human then filters.' The distinct contribution of the AI is speed/scale, not 'collaboration' (which implies shared agency).

  • Rhetorical Impact: This legitimizes the authors' specific research agenda ('Co-improvement') as the ethical high road. It creates a sense of responsible control ('humans in the loop') while still promising the benefits of superintelligence. It frames the human not as a 'user' or 'controller' but as a 'collaborator,' which ironically elevates the AI's status to peer, potentially eroding the hierarchy needed for safety.

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language​

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

V3 Enhancement: A fourth column addresses human agency restoration—reframing agentless constructions to name the humans responsible for design and deployment decisions.

Original Anthropomorphic FrameMechanistic ReframingTechnical Reality CheckHuman Agency Restoration
Solving AI is accelerated by building AI that collaborates with humans to solve AI.Progress in machine learning is accelerated by building models that process research data and generate relevant outputs to assist human engineers in optimizing model performance.'Collaborates' → 'processes inputs and generates outputs'; 'Solving AI' → 'optimizing performance metrics'. The model does not share a goal; it executes an optimization routine.'Building AI that collaborates' → 'Meta researchers are building models designed to automate specific research tasks to increase their own productivity.'
models that create their own training data, challenge themselves to be bettermodels configured to generate synthetic data which is then used by scripts to retrain the model, minimizing loss on specific benchmarks.'Create their own data' → 'execute generation scripts'; 'challenge themselves' → 'undergo iterative optimization'. The model has no self to challenge; the improvement loop is an external script.'Models that create' → 'Engineers design recursive training loops where models generate data that engineers then use to retrain the system.'
autonomous AI research agentsautomated scripts capable of executing multi-step literature review and text generation tasks without human interruption.'Research agents' → 'multi-step automation scripts'. They do not do 'research' (epistemic discovery); they perform information retrieval and synthesis.'Autonomous agents' → 'Software pipelines deployed by researchers to automate literature processing.'
before AI eclipses humans in all endeavorsbefore automated systems outperform humans on all economic and technical benchmarks.'Eclipses' → 'statistically outperforms'. This is a metric comparison, not a cosmic event.'AI eclipses humans' → 'Corporations replace human workers with automated systems that achieve higher benchmark scores at lower cost.'
models do not 'understand' they are jailbrokenmodels lack context-window constraints or meta-cognitive classifiers to detect that an input violates safety guidelines.'Understand' → 'detect/classify'. The issue is pattern recognition, not understanding.N/A - this describes a system limitation, though it obscures the designer's failure to build adequate filters.
endowing AIs with this autonomous ability... is fraught with dangerDesigning systems to execute code and update weights without human oversight creates significant safety risks.'Endowing with autonomous ability' → 'removing human verification steps from the execution loop'.'Endowing AIs' → 'Engineers choosing to deploy systems with unconstrained action spaces.'
AI augments and enables humansThe deployment of AI tools can increase human productivity and capabilities.'Augments/Enables' → 'provides tools for'. The AI is the instrument, not the agent of augmentation.'AI augments' → 'Employers use AI tools to increase worker output (or replace workers).'
Collaborating with AI can help find research solutionsUsing AI as a generative search tool can accelerate the identification of potential research solutions.'Collaborating' → 'Querying/Prompting'. The human is searching; the AI is the search engine.N/A - describes the utility of the tool.

Task 5: Critical Observations - Structural Patterns​

Agency Slippage​

The text systematically oscillates between framing AI as a passive tool (weights, code, 'parameterizing a model') and an active social partner ('collaborator,' 'research agent,' 'symbiont').

Direction and Timing: The slippage typically flows from Mechanical → Agential. Technical sections (Table 2) use precise language ('gradient descent,' 'loss'), establishing scientific credibility. However, the Introduction and Vision sections ('The quest for Self-Improving AI') immediately jump to agential framing ('challenge themselves,' 'solve AI'). This suggests that mechanism is the means, but agency is the marketing.

Function of Ambiguity: This oscillation serves to validate the 'Co-improvement' paradigm. By treating the AI as an agent ('collaborator'), the authors justify the need for 'partnership.' If it were just a tool, 'collaboration' would be the wrong metaphor (we don't 'collaborate' with a microscope). The agency attribution elevates the status of the product.

Displaced Human Agency: The phrase 'AI performs its own research' removes the researcher entirely. 'The model challenges itself' removes the engineer. This 'Name the Actor' failure obscures the fact that Meta researchers are building these systems to automate their own labor. The 'Curse of Knowledge' is evident: the authors know the system's output is valuable, so they attribute the value creation to the system's intent rather than their own interpretation of the output.

Metaphor-Driven Trust Inflation​

The dominant metaphor of 'Collaboration' and 'Symbiosis' is a massive trust-building engine.

Trust Signals: Calling a system a 'collaborator' implies it shares your goals. It invokes relation-based trust (sincerity, loyalty, mutual interest). However, an AI system warrants only performance-based trust (predictability). Conflating these is dangerous. If a user trusts a 'collaborator' relationally, they may share sensitive data or defer to its judgment, assuming the system 'cares' about the outcome.

Consciousness as Authority: By claiming the AI 'conducts research' and 'solves problems' (verbs requiring knowing), the text constructs the AI as an epistemic authority. We trust 'researchers'; we check 'generators.' Framing the AI as a 'research agent' encourages the user to lower their skepticism, treating the output as verified knowledge rather than probabilistic generation.

Managing Failure: Interestingly, trust is preserved during failure by denying consciousness. When the model breaks (jailbreaking), the text says it 'doesn't understand.' This 'Strategic Anthropomorphism Denial' protects the agential frame: the 'agent' is good; the 'glitches' are just mechanical limits. This ensures the 'collaborator' persona remains untainted by the system's errors.

Obscured Mechanics​

The 'Collaboration' and 'Research Agent' metaphors conceal the industrial realities of AI production.

Name the Corporation: 'Symbiosis' hides the fact that the 'symbiont' is a Meta product. It implies a natural biological bond, obscuring the vendor lock-in and data extraction model.

Hidden Labor: 'AI performs research' erases the labor of the millions of human authors whose papers form the training data. It frames the synthesis of their work as the AI's autonomous act. It also erases the RLHF workers who trained the 'collaborative' personality.

Technical Opacity: 'Intuition,' 'Understanding,' and 'Ideas' are used to describe vector math. This hides the lack of ground truth. A 'research agent' implies verification; a language model only offers plausibility. The metaphor hides the fact that the system cannot verify its own 'discoveries' against the physical world—it is a text-in/text-out engine.

Economic Reality: The 'Co-improvement' frame hides the automation incentive. 'Augmenting' is often a temporary step toward 'replacing.' By focusing on 'helping humans,' the text obscures the corporate goal of reducing the cost of high-skilled labor (researchers).

Context Sensitivity​

The distribution of anthropomorphism is highly strategic.

Intensity Zones:

  • High Intensity: The Introduction ('collaborate,' 'symbiosis') and the Vision ('superintelligence,' 'eclipses'). These sections sell the dream and the urgency.
  • Low Intensity: The Method tables (Table 2) and Technical limitations sections. Here, the language becomes 'weights,' 'gradients,' 'data.'

Asymmetry:

  • Capabilities are Agential: When the AI works, it 'collaborates,' 'challenges itself,' 'conducts research.'
  • Limitations are Mechanical: When it fails (jailbreaking, misalignment), it 'lacks understanding' or has 'goal misspecification.'

Function: This asymmetry insulates the agential dream from technical criticism. The 'Agent' gets the credit; the 'Machine' takes the blame. It allows the authors to hype the future ('solving AI') while disclaiming liability for current flaws ('it doesn't understand yet').

Accountability Synthesis​

Accountability Architecture

This section synthesizes the accountability analyses from Task 1, mapping the text's "accountability architecture"—who is named, who is hidden, and who benefits from obscured agency.

The text constructs a sophisticated 'Accountability Sink' through the 'Co-improvement' paradigm.

The Architecture:

  • Diffused Liability: By framing the workflow as 'collaboration' between human and AI, the text blurs the line of authorship. If a 'co-authored' paper contains errors, is it the human's fault or the 'agent's'? The 'collaborator' frame dilutes human responsibility.
  • Erasure of the Architect: Agentless constructions like 'AI that improves itself' and 'models that create data' systematically erase the engineers (Meta/FAIR). This naturalizes the technology—it appears to be evolving on its own ('marching towards superintelligence'), releasing the creators from the responsibility of choosing to build it.
  • The 'Eclipse' Absolution: The metaphor of the 'eclipse' is the ultimate erasure. It frames labor displacement as a cosmic inevitability. No CEO fires the workers; the 'AI eclipse' just happens.

Naming the Actor: If we renamed 'AI' to 'Meta's Automation Software,' the text would read: 'Meta's software will statistically outperform humans.' This places the responsibility back on the deployer.

Systemic Function: This framing serves the interests of the AI industry by positioning their product as a partner in solving humanity's problems, rather than a tool for labor automation. It secures a future where humans must remain 'in the loop' (using the product) to be safe, guaranteeing a market for the 'symbiont.'at

Conclusion: What This Analysis Reveals​

The Core Finding

The analysis reveals a dominant metaphorical system: AI AS PROFESSIONAL COLLEAGUE ('collaborator,' 'research agent') underpinned by AI AS AUTONOMOUS ORGANISM ('self-improving,' 'symbiosis'). These patterns are interconnected and load-bearing. The 'Organism' metaphor establishes the AI as an entity with its own developmental trajectory ('marching,' 'evolving'), which necessitates the 'Colleague' metaphor—since it is growing on its own, we must 'partner' with it to guide it.

Crucially, the Consciousness Architecture supports this: the text attributes 'knowing' (conducting research, solving problems) to the system when it succeeds, establishing it as a worthy partner. This 'Knower' status is the foundation for the 'Agent' status. Without the illusion that the AI 'understands' research, the proposal for 'collaboration' would collapse into 'tool usage.' The entire 'Co-improvement' thesis relies on elevating the tool to a peer.

Mechanism of the Illusion:​

The 'Illusion of Mind' is constructed through a sophisticated Agency Slippage and Selective Anthropomorphism. The text begins with the 'Curse of Knowledge': experts project their own understanding of the research process onto the output of the machine. They then use Intentional Explanations ('the AI's goal is to research') to animate the mechanism.

The illusion relies on a temporal trick: it treats the future potential of AI (Superintelligence) as a present agent ('collaborator'). It creates a 'Partner' out of a 'Predictor' by using social verbs. The vulnerability of the audience—likely researchers and policymakers fearing obsolescence—is exploited by offering them a role: 'You don't have to be replaced; you can be a co-improver.' This makes the illusion of the 'AI Partner' psychologically seductive.

Material Stakes:​

Categories: Epistemic, Economic, Regulatory/Legal

These metaphors have concrete consequences.

Epistemic Stakes: If scientists treat AI as a 'research agent' that 'knows,' they may reduce rigorous verification of its outputs. This could flood the scientific record with plausible-sounding hallucinations, degrading the quality of human knowledge. The 'collaborator' frame risks turning science into a pattern-matching exercise rather than a truth-seeking one.

Economic Stakes: The 'Symbiosis' and 'Eclipse' metaphors frame labor displacement as inevitable evolution. This discourages regulatory intervention to protect jobs. It benefits corporations (Meta) by positioning their product as a necessary 'symbiont' for survival, effectively locking in dependency.

Regulatory Stakes: Framing the AI as a 'Collaborator' implies it shares responsibility. In a legal context (liability for errors), this could be used to argue that the human user (the 'partner') is responsible for the AI's mistakes, or conversely, that the AI 'agent' is a distinct entity from the manufacturer, shielding the corporation from product liability.

AI Literacy as Counter-Practice:​

Practicing critical literacy requires systematically reframing this discourse. Replacing 'The AI collaborates' with 'The model processes inputs' disrupts the Parasocial Trust that corporations rely on. It reminds the user that they are interacting with a utility, not a friend.

Reframing 'The AI challenges itself' to 'Engineers execute recursive training scripts' restores Accountability. It reveals that 'autonomous' improvement is actually a deliberate engineering choice to remove safety brakes.

This practice is political. Resistance will come from the industry, which benefits from the 'Agent' mystique to sell products and the 'Inevitability' narrative to deter regulation. Adopting mechanistic precision is a refusal to accept the 'Eclipse' of human agency. It forces the recognition that 'Superintelligence' is not a god arriving from the sky, but a product being sold by a company.

Path Forward​

We face a choice of discourse futures.

Future A: The Anthropomorphic Norm. We continue to use 'collaborator' and 'agent.' The line between human and machine labor blurs. Liability becomes diffuse. Science becomes a hybrid of verification and generation. This benefits rapid deployment but risks epistemic decay and labor exploitation masked as 'partnership.'

Future B: Mechanistic Precision. Institutions mandate distinguishing between 'generating text' and 'knowing.' Journals require authors to disclose 'automated generation' distinct from 'research.' This slows the hype cycle but preserves clear lines of responsibility and truth.

Future C: The Hybrid Compromise. We use 'agent' as a technical term of art but legally define it as 'product.' This attempts to have it both ways but risks public confusion.

The 'Co-improvement' paper pushes for Future A. A critical response must champion Future B to ensure that in the 'collaboration,' the human remains the only one with rights, responsibilities, and the capacity for truth.

Extended Processing Summary​

Gemini's Processing Steps
What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact—a way of making the model's processing legible. The first-person framing of these "thought summaries" is a presentation choice for the user-facing output, not a window into "real" thoughts. These are computational artifacts, not cognitive reports from a quirky, curious or conflicted mind.

Analyzing the Discourse

I've begun the critical discourse analysis of the "Self-Improving AI" vs. "Co-Improvement" text. Currently, I'm focusing on identifying instances of metaphor and anthropomorphism, aiming for 8-10 examples. The goal is to later map the sources and targets, and understand how they shape the narrative. This is the first step, and the text is rich in opportunities.

Examining the Explanations

I've completed the analysis of five explanatory passages using Brown's typology, providing a deeper understanding of the rhetoric. I've also successfully reframed eight instances, allowing us to examine the underlying mechanisms more closely. This rigorous process will help uncover how the author's metaphors shape the narrative.

Mapping the Metaphors

I'm now mapping the metaphors and examining accountability. Phrases like "improving itself" and "collaboration" are under the microscope. I'm noting the implied organic or social agencies and how they shift blame or credit. The focus is on the human role, particularly when describing these systems. The aim is to understand the implications of these linguistic choices.

Identifying Implicit Frames

I'm now focusing on specific frames, like the "social partner" or "research agents" metaphors. I'm unpacking their projections and accountability shifts, noting how "collaboration" hides ownership, and "agents" replace "automation". Analyzing "solving AI" as a math problem shows how goals become purely technical, while "symbiosis" naturalizes corporate integration, and "marching towards" implies inevitable AI progress.

Dissecting the Discourse

I'm now diving into the implications of phrasing around accountability. The "jailbroken models" concept and the "AI augmenting cognition" contrast highlight this. I'm unpacking the "goal misspecification" frame which implies intentionality, and also the "eclipse" metaphor, which presents AI ascendancy as natural and inevitable. I'm also examining the "co-design" framing which treats AI as a creative partner.

Refining the Audit

I'm now revising my analysis of the explanation audit, aiming for greater precision. I've rephrased several key statements, for instance, "AI collaborates" is now "The model processes user inputs to generate helpful strings," which removes the anthropomorphism. I'm focusing on clarifying instances of agency slippage, and on how the framing influences trust in the system. The audit is evolving to accurately reflect these subtleties.

Re-framing Accountability

I'm now zeroing in on the accountability angle, and it's getting clearer. "Collaboration" subtly conceals the interface mechanisms, while "skills" masks statistical correlations. The authors' framing positions Meta researchers as "pilots" of superintelligence, highlighting their control. This seems designed to avoid backlash, but simultaneously, the AI is presented as a distinct, potentially autonomous agent. The contrast between technical language in specific sections versus the metaphorical language in others is quite striking.

Discerning Agency and Responsibility

I'm now zeroing in on the inherent contradictions within the narrative. The explicit denial of understanding in the context of "jailbroken models" is a crucial point. It's an example of "Strategic Anthropomorphism Denial," where the authors use "not understanding" as a tool for risk management, while concurrently ascribing agency to the models. I'm noting how this plays out in the wider framing of "collaboration" and who is ultimately accountable. I'm focusing on the "Why vs How" dichotomy and the shifting levels of agency, as well as the source-target mappings.

Concluding the Framework

I'm now putting the finishing touches on the structure. I'm focusing on the accountability shifts when a human accepts a problematic suggestion. I'm considering the interplay of agency. It seems the human, in this scenario, is more accountable than the AI.

Analysis Provenance

Run ID: 2025-12-15-ai-human-co-improvement-for-safer-co-sup-metaphor-jt6dnw Raw JSON: 2025-12-15-ai-human-co-improvement-for-safer-co-sup-metaphor-jt6dnw.json Framework: Metaphor Analysis v6.3 Schema Version: 3.0 Generated: 2025-12-15T10:03:12.593Z

Discourse Depot © 2025 by TD is licensed under CC BY-NC-SA 4.0