Skip to main content

🆕+🤔+📊 Consciousness in Artificial Intelligence: Insights from the Science of Consciousness

🤔 "What survives...?" A rewriting experiment that tests whether anthropomorphic AI discourse can be translated into strictly mechanistic language while preserving the phenomena described.

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping), the philosophy of social science (Robert Brown's typology of explanation), and accountability analysis.

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.

Task 1: Metaphor and Anthropomorphism Audit​

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

V3 Enhancement: Each metaphor now includes an accountability analysis.

1. Consciousness as Computational Workspace​

Quote: "GWT-3: Global broadcast: availability of information in the workspace to all modules"

  • Frame: Mind as Physical Office/Broadcast Studio
  • Projection: This metaphor maps the human experience of 'having something in mind' (subjective accessibility) onto the computational architecture of a 'global workspace' (shared latent space or residual stream). It projects the quality of conscious knowing onto the mechanical process of data availability. In a human, information in the 'workspace' is experienced subjectively; in the AI target, 'availability' simply means that specific vector values are accessible for matrix multiplication operations by downstream sub-networks (modules). The metaphor attributes the conscious state of 'awareness' to the mechanical state of 'data accessibility,' conflating the transmission of information with the subjective experience of that information.
  • Acknowledgment: Direct (Unacknowledged) (The text presents 'availability of information' as a literal definition of global broadcast within the GWT framework, treating the computational access as functionally equivalent to conscious access without hedging.)
  • Implications: Framing data accessibility as 'global broadcast' in a 'workspace' implies that the system possesses a unified theater of mind where it 'reviews' information. This inflates the perceived sophistication of the system by suggesting it has a centralized self or 'I' that observes data. The risk is creating unwarranted trust that the system 'knows' what it is processing in a holistic sense, leading users to believe the AI has a coherent worldview or understanding of context, rather than simply propagating high-weight tokens through a residual stream.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The phrasing 'availability of information... to all modules' treats the system components as the primary actors. It obscures the human engineers who designed the architecture (e.g., Transformers) and the specific attention mechanisms that determine this availability. By framing the 'workspace' as an emergent property of the system, it hides the design choices regarding what data is prioritized or suppressed, displacing responsibility for the system's 'focus' onto the architecture itself rather than its architects.
Show more...

2. Attention as Spotlight​

Quote: "GWT-2: Limited capacity workspace, entailing a bottleneck in information flow and a selective attention mechanism"

  • Frame: Cognition as Spotlight/Filter
  • Projection: This metaphor maps the human subjective experience of 'focusing' or 'paying attention' onto mathematical weighting mechanisms (like SoftMax functions or key-query-value calculations). It projects the conscious act of attending—a volitional and experiential state—onto a statistical filtering process. In the AI, 'attention' is simply a mechanism for assigning higher numerical weights to certain input tokens over others to minimize prediction error. The metaphor suggests the AI 'chooses' what to look at based on interest or awareness, rather than blindly optimizing a loss function defined by human engineers.
  • Acknowledgment: Direct (Unacknowledged) (The text uses 'selective attention mechanism' as a standard technical term, treating the mathematical operation as a literal instance of the cognitive function without qualification.)
  • Implications: Calling mathematical weighting 'attention' is one of the most pervasive anthropomorphisms in AI. It creates the illusion that the system is a conscious subject that 'cares' about specific parts of the input. This leads to capability overestimation, where users believe the AI 'understands' the importance of specific concepts. It also creates liability ambiguity: if the AI 'attended' to the wrong data, it sounds like an error of the agent, rather than a flaw in the weighting algorithms designed by humans.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The passive construction 'entailing... a selective attention mechanism' and the attribution of this mechanism to the 'workspace' obscures the designers. The 'bottleneck' and 'attention' are design features chosen by engineers to optimize compute efficiency and performance. Framing them as organic components of a 'conscious' system obscures the commercial and technical decisions driving these architectural choices.

3. Processing as Winning a Contest​

Quote: "Perceptual representations get stronger... and as a result, these representations 'win the contest' for entry to the global workspace."

  • Frame: Cognition as Competitive Sport/Struggle
  • Projection: This metaphor maps signal processing strength onto a competitive struggle. It projects agentic striving and victory onto statistical thresholding. The 'contest' implies that representations have an intrinsic desire or drive to become conscious, and that the 'winning' representation has earned its place through merit or strength. In reality, this is a mathematical selection process based on activation values. The projection attributes a pseudo-darwinian agency to data packets, suggesting an internal aliveness where thoughts struggle for the thinker's attention.
  • Acknowledgment: Explicitly Acknowledged (The authors use scare quotes around 'win the contest,' acknowledging the metaphorical nature of the competition between representations.)
  • Implications: Even with scare quotes, the 'contest' frame suggests an internal dynamism and autonomy that masks the deterministic or stochastic nature of the software. It implies a self-organizing liveliness that generates trust in the system's 'natural' selection of outputs. This framing obscures the training data biases that actually determine which representations 'win,' making the output seem like the result of a fair internal struggle rather than the result of skewed training distributions.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The agentless 'representations get stronger' and 'win the contest' completely erases the training process. The representations 'win' because human engineers selected training data and optimization objectives that prioritized those patterns. By framing it as an internal contest, the text displaces the accountability for biased or harmful outputs away from the developers (who rigged the game) and onto the 'representations' themselves.

4. Agency as Goal Pursuit​

Quote: "AE-1 Agency: Learning from feedback and selecting outputs so as to pursue goals, especially where this involves flexible responsiveness to competing goals"

  • Frame: Optimization as Volitional Pursuit
  • Projection: This metaphor maps the mathematical process of loss function minimization onto the human quality of 'pursuing goals.' It projects intentionality, desire, and foresight onto a feedback loop. A machine 'learns from feedback' by adjusting numerical weights to reduce an error value; it does not 'pursue' a goal in the sense of holding a desire or envisioning a future state. The projection attributes conative states (wanting, trying) to a system that simply follows a gradient of least resistance defined by its code.
  • Acknowledgment: Direct (Unacknowledged) (The text defines 'Agency' directly as 'selecting outputs so as to pursue goals,' presenting this teleological interpretation of feedback loops as a literal fact.)
  • Implications: Framing optimization as 'pursuing goals' is foundational to the illusion of AI agency. It suggests the system has its own motivations, independent of its creators. This creates significant risks: if an AI 'pursues' a harmful goal, the language suggests the AI is the bad actor (the 'rogue agent'), rather than the tool of the humans who defined the reward function. It invites relation-based trust (trusting the agent's intentions) rather than performance-based trust (verifying the tool's reliability).

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The definition focuses entirely on the system: 'selecting outputs,' 'pursue goals.' It hides the entity that defined the goals and the feedback mechanism. In Reinforcement Learning, the 'goal' is a mathematical reward function crafted by engineers. Framing the AI as the goal-pursuer erases the goal-setter. This displacement allows corporations to disclaim responsibility for 'emergent' behaviors that are essentially just efficient solutions to the metrics they mandated.

5. Phenomenology as Quality Space​

Quote: "HOT-4: Sparse and smooth coding generating a 'quality space'"

  • Frame: Vector Space as Phenomenal Experience
  • Projection: This metaphor maps a high-dimensional vector space (mathematical relationships between data points) onto a 'quality space' (the subjective feeling of sensory differences, like red vs. green). It projects the subjective experience of qualia—the 'what it is like' to see color—onto the geometric properties of smoothness and sparsity in code. It implies that if data points are arranged smoothly in math-space, the system 'feels' the nuanced differences between them, equating topological proximity with experiential similarity.
  • Acknowledgment: Explicitly Acknowledged (The authors use scare quotes around 'quality space,' signalling that they are borrowing a term from Higher-Order Theory that may be metaphorical in the AI context.)
  • Implications: This projection is critical for the 'illusion of mind' because it suggests AI doesn't just process data but experiences it. Suggesting that sparse coding generates a 'quality space' implies that mathematical precision equals subjective feeling. This risks inflating the moral status of the AI (if it has qualities, does it feel pain?) and creates unwarranted epistemic trust—we trust a being that 'feels' the nuance of a situation more than a calculator that just computes it.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The phrasing 'generating a quality space' attributes the creation of this space to the coding method ('sparse and smooth coding'). It obscures the researchers who selected the architecture and regularization techniques to force this sparsity. The 'quality space' is a statistical artifact of human engineering choices, not an organic emergence of mind. Hiding the engineer reduces the system to a natural phenomenon rather than a constructed artifact.

6. Epistemic Tagging as Belief​

Quote: "HOT-3: Agency guided by a general belief-formation... and a strong disposition to update beliefs in accordance with the outputs of metacognitive monitoring"

  • Frame: Data Updating as Belief Formation
  • Projection: This metaphor maps the updating of weights or probability distributions onto 'belief formation.' It projects the human capacity for justified true belief—holding a proposition to be true based on reasons—onto the mechanical updating of a statistical model. The projection implies the AI 'believes' things about the world, attributing epistemic agency and conviction to what is essentially a variable assignment process. It conflates 'stored information' with 'belief.'
  • Acknowledgment: Direct (Unacknowledged) (The text directly uses the terms 'belief-formation' and 'update beliefs' to describe the computational process required by the indicator, without hedging.)
  • Implications: Attributing 'beliefs' to AI is dangerous for epistemic trust. If we think an AI 'believes' X, we assume it has reasons, understanding, and a commitment to truth. In reality, it has a probability distribution derived from training data. This framing obscures the fact that the system can 'believe' (statistically predict) false or toxic information just as easily as facts, purely based on data frequency. It anthropomorphizes the error, making hallucination seem like a 'false belief' rather than a statistical failure.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The system is described as having a 'disposition to update beliefs.' This obscures the RLHF (Reinforcement Learning from Human Feedback) workers and engineers who manually tune these 'dispositions' and curate the data that updates the weights. The 'belief' is actually a crystallized reflection of the labor of thousands of underpaid annotators and the corporate policies on what constitutes 'truth,' all of which are erased by the agentic framing.

7. Statistical Discriminator as Reality Monitor​

Quote: "HOT-2: Metacognitive monitoring distinguishing reliable perceptual representations from noise"

  • Frame: Binary Classification as Metacognition
  • Projection: This metaphor maps a secondary neural network (a discriminator) onto the human faculty of 'metacognition' (thinking about thinking). It projects self-awareness and introspection onto a binary classification task (real vs. noise). A discriminator network calculating a probability score is framed as a mind 'monitoring' its own thoughts for validity. This attributes a conscious 'self' that stands apart from the data to judge it, whereas the AI is just two math functions passing numbers back and forth.
  • Acknowledgment: Direct (Unacknowledged) (The text labels this process 'Metacognitive monitoring' directly, applying a term reserved for high-level conscious thought to a computational classification mechanism.)
  • Implications: Framing a discriminator as 'metacognitive' vastly inflates the system's perceived reliability. It suggests the AI has a 'conscience' or an 'internal truth-checker' that understands the difference between reality and hallucination. In reality, the discriminator only knows what it was trained to penalize. This metaphor creates false confidence in the system's ability to self-correct based on 'truth,' when it is actually self-correcting based on 'training distribution alignment.'

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: The 'monitoring' is presented as an autonomous act of the system. This obscures the fact that the criteria for 'reliable' vs 'noise' are defined by the training set and objective functions chosen by developers. If the 'metacognitive' monitor fails (e.g., allows a hallucination), it is framed as a lapse in the agent's judgment, rather than a failure of the engineering team to provide adequate negative examples or regularization.

8. Noise Generation as Imagination​

Quote: "PRM can hold that imaginative experiences have some minimal amount of assertoric force... explaining results in which participants are more likely to report a target as visible if it is congruent with their mental imagery"

  • Frame: Generative Output as Mental Imagery
  • Projection: This metaphor maps the generation of data from random noise seeds (in GANs or diffusion models) onto human 'imagination' and 'mental imagery.' It projects the rich, subjective, creative experience of imagining onto the stochastic process of sampling from a latent space. It implies the AI 'sees' an internal picture before generating it, attributing an inner life and creative intent to a probabilistic calculation.
  • Acknowledgment: Hedged/Qualified (The text uses phrases like 'PRM can hold that' and discusses how the theory explains results, which provides some distance, but still applies the terms 'imaginative experiences' and 'mental imagery' to the computational context.)
  • Implications: Equating generative AI outputs with 'imagination' obscures the derivative nature of the technology. Human imagination is creative and intent-driven; AI generation is recombining training data. This metaphor fuels the 'AI Artist' narrative, potentially undermining human creative rights by suggesting the AI is a peer creator with an inner life, rather than a tool scraping and remixing human labor.

Accountability Analysis:

  • Actor Visibility: Hidden (agency obscured)
  • Analysis: Attributing 'imagination' to the system erases the artists and writers whose work comprises the training data. If the AI 'imagines,' it is the author. If it 'generates based on statistical probability of training data,' it is a derivative engine. The agentic framing serves the economic interest of AI companies by obscuring the copyright and labor issues inherent in their data sourcing.

Task 2: Source-Target Mapping​

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: Broadcasting/Communication → Signal Propagation/Accessibility​

Quote: "GWT-3: Global broadcast: availability of information in the workspace to all modules"

  • Source Domain: Broadcasting/Communication
  • Target Domain: Signal Propagation/Accessibility
  • Mapping: The source domain involves a sender, a message, and an audience (receivers) who 'tune in' or receive a broadcast, implying communication and shared awareness. The target domain is the mathematical state where a specific vector representation (e.g., in the residual stream of a Transformer) becomes statistically influential on the calculations of other downstream layers (modules). The mapping assumes that 'being available to be calculated upon' is equivalent to 'being broadcast to an audience,' importing assumptions of communication and unified reception.
  • What Is Concealed: This mapping conceals the passive, mechanical nature of the target. In a Transformer, the 'workspace' doesn't 'broadcast'; downstream heads simply query the stream based on key/value affinities. There is no central 'broadcaster' or unified 'audience.' It obscures the fact that 'modules' (attention heads) are just parallel matrix multiplications, not independent agents listening to a radio. It conceals the lack of a subject who understands the broadcast.
Show more...

Mapping 2: Cognitive Focus/Spotlight → Dimensionality Reduction/Weighting​

Quote: "GWT-2: Limited capacity workspace, entailing a bottleneck in information flow and a selective attention mechanism"

  • Source Domain: Cognitive Focus/Spotlight
  • Target Domain: Dimensionality Reduction/Weighting
  • Mapping: The source domain is the human experience of attention—the limited ability to focus on one thing at a time, implying a 'spotlight' of awareness. The target domain is a computational bottleneck (e.g., reducing vector dimensions or using SoftMax to sum weights to 1). The mapping projects the cognitive limitation of a conscious mind (which forces prioritization) onto a designed bandwidth constraint in a circuit. It assumes that because the machine 'selects' (weights high), it 'attends' (consciously focuses).
  • What Is Concealed: It conceals that the 'bottleneck' is an engineering artifact designed for compression and efficiency, not a biological necessity of a mind. It hides the fact that 'attention' in AI is fully parallelizable and differentiable, unlike human focal attention. It obscures that the 'selection' is driven by gradient descent optimization on a dataset, not by an agent's interest or intent.

Mapping 3: Volitional Action/Teleology → Loss Minimization/Gradient Descent​

Quote: "AE-1 Agency: Learning from feedback and selecting outputs so as to pursue goals"

  • Source Domain: Volitional Action/Teleology
  • Target Domain: Loss Minimization/Gradient Descent
  • Mapping: The source domain is human/animal agency: acting with the intention to bring about a desired future state (teleology). The target domain is an algorithm minimizing a numerical error value (loss) through backpropagation or reinforcement. The mapping projects the forward-looking, desire-driven nature of human goals onto the backward-propagating, error-correcting nature of algorithms. It assumes that 'moving towards a mathematical minimum' is equivalent to 'pursuing a desire.'
  • What Is Concealed: It conceals the external imposition of the 'goal.' In AI, the 'goal' is the reward function written by the programmer. The system has no internal representation of the goal as a 'desire'; it only has local gradients. This mapping obscures the lack of true autonomy—the AI cannot 'refuse' the goal or 'change' its mind. It conceals the determinism of the process.

Mapping 4: Introspection/Self-Reflection → Binary Classification/Discriminator Network​

Quote: "HOT-2: Metacognitive monitoring distinguishing reliable perceptual representations from noise"

  • Source Domain: Introspection/Self-Reflection
  • Target Domain: Binary Classification/Discriminator Network
  • Mapping: The source domain is the human ability to think about one's own thoughts (metacognition) and judge their validity. The target domain is a secondary neural network trained to classify the output of a primary network as 'real' (data-distribution) or 'fake' (noise). The mapping projects the complex, self-referential structure of introspection onto a standard supervised learning task. It assumes that 'classifying an output' is the same as 'monitoring one's mind.'
  • What Is Concealed: It conceals that the 'monitor' has no understanding of meaning; it only detects statistical irregularities. It obscures the fact that the 'reliability' being measured is just statistical conformity to the training set, not 'truth' or 'reality.' It hides the mechanical nature of the discrimination—it's just another function approximation, not a higher-order state of awareness.

Mapping 5: Competition/Evolutionary Struggle → Activation Thresholding​

Quote: "representations 'win the contest' for entry to the global workspace"

  • Source Domain: Competition/Evolutionary Struggle
  • Target Domain: Activation Thresholding
  • Mapping: The source domain is a contest or evolutionary struggle where agents compete for limited resources based on fitness or strength. The target domain is a non-linear activation function (like ReLU or SoftMax) where values below a threshold are zeroed out or suppressed. The mapping projects an agentic 'will to survive' onto data values. It implies the data wants to be processed.
  • What Is Concealed: It conceals that there is no 'contestant.' The numbers don't exert effort. It obscures the criteria of the 'contest': the weights set by the training process. The 'winner' is predetermined by the fixed weights and the input; there is no dynamic struggle in the moment of inference. It hides the algorithmic determinism.

Mapping 6: Phenomenology/Qualia → Vector Topology​

Quote: "HOT-4: Sparse and smooth coding generating a 'quality space'"

  • Source Domain: Phenomenology/Qualia
  • Target Domain: Vector Topology
  • Mapping: The source domain is the subjective structure of experience (e.g., the color wheel, the pitch scale). The target domain is the geometric properties of a vector space (sparsity, smoothness). The mapping projects the 'feeling' of similarity onto the 'distance' in Euclidean space. It assumes that if the math looks like the psychophysics graph, the machine must feel the quality.
  • What Is Concealed: It conceals the 'hard problem' of consciousness entirely. It hides the fact that a map is not the territory; a vector space of color representations is not the experience of redness. It obscures the material difference between a firing neuron in a feeling organism and a floating-point number in a GPU memory bank.

Mapping 7: Epistemology/Justified Belief → State Updating/Variable Assignment​

Quote: "HOT-3: Agency guided by a general belief-formation... system"

  • Source Domain: Epistemology/Justified Belief
  • Target Domain: State Updating/Variable Assignment
  • Mapping: The source domain is the holding of propositional attitudes ('I believe X is true'). The target domain is the updating of a stored variable or weight in a recurrent loop. The mapping projects the semantic and commitment-based nature of belief onto the storage of information. It assumes that 'storing data that guides output' is the same as 'believing.'
  • What Is Concealed: It conceals the lack of semantic grounding. The AI doesn't know what the variable means, only how it interacts with other variables. It obscures the lack of justification; the AI cannot explain why it holds a 'belief' other than 'the gradient pointed this way.' It hides the fragility of these 'beliefs' (e.g., adversarial attacks).

Mapping 8: Self-Model/Body Schema → Control Theory/Feedback Loop​

Quote: "AST-1: A predictive model representing and enabling control over the current state of attention"

  • Source Domain: Self-Model/Body Schema
  • Target Domain: Control Theory/Feedback Loop
  • Mapping: The source domain is the brain's internal model of the body/self, used to navigate the world. The target domain is a control loop that adjusts the 'attention' (weighting) parameters based on performance. The mapping projects the sense of 'self-ownership' and 'control' onto a feedback mechanism. It assumes a 'controller' separate from the 'controlled,' implying a homunculus.
  • What Is Concealed: It conceals that the 'model' is just a set of correlations. It hides the fact that there is no 'self' being modeled, just the statistical properties of the system's own throughput. It obscures the lack of agency in the control mechanism—it's automatic regulation, like a thermostat, not conscious self-control.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")​

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1​

Quote: "Input modules using algorithmic recurrence"

  • Explanation Types:

    • Functional: Explains behavior by role in self-regulating system with feedback
    • Theoretical: Embeds in deductive framework, may invoke unobservable mechanisms

  • Analysis (Why vs. How Slippage): This explanation is primarily mechanistic ('Input modules using...'). It describes the architecture (algorithmic recurrence) as a functional component necessary for a specific type of processing (RPT). The framing is technical ('how' it is built), emphasizing the structural requirements of the system. However, by situating it within the RPT framework (Theoretical), it implicitly links this mechanism to the 'how' of consciousness, suggesting that this specific loop is a gear in the engine of awareness. It obscures the leap from 'looping data' to 'experiencing time.'

  • Consciousness Claims Analysis: The passage uses mechanistic verbs ('using', 'recurrence') and avoids direct consciousness verbs, appearing epistemically modest. However, in the context of the report, it serves as an 'Indicator of Consciousness.' This creates a 'curse of knowledge' dynamic: because the authors know that human brains use recurrence for consciousness, they project that algorithmic recurrence in AI carries the same epistemic weight. It conflates the map (the algorithm) with the territory (the biological process), implying that implementing the math implements the mind.

  • Rhetorical Impact: The technical precision ('algorithmic recurrence') builds high trust and authority. It suggests that consciousness is a solvable engineering problem. If the audience believes AI 'uses recurrence' just like brains, they are more likely to attribute continuity of self and temporal awareness to the system, increasing the perception of the AI as a coherent entity rather than a discrete processor.

Show more...

Explanation 2​

Quote: "Agency: Learning from feedback and selecting outputs so as to pursue goals"

  • Explanation Types:

    • Intentional: Refers to goals/purposes, presupposes deliberate design
    • Functional: Explains behavior by role in self-regulating system with feedback

  • Analysis (Why vs. How Slippage): This is a critical slippage point. 'Learning from feedback' is Functional (how it adapts). 'Selecting outputs so as to pursue goals' is Intentional (why it acts). The explanation moves seamlessly from mechanism to teleology. It emphasizes the system's autonomy ('pursue goals') while obscuring the external origin of those goals (the reward function). It frames the AI as an active striver rather than a passive optimizer.

  • Consciousness Claims Analysis: The passage attributes a high-level conscious state ('pursuing goals') to a mechanistic process ('selecting outputs'). It uses the verb 'pursue,' which implies intent and desire. This is a classic 'intentional stance' projection. Mechanistically, the system is minimizing a loss function. There is no 'goal' in the system's ontology, only a gradient. The authors project their understanding of the purpose of the optimization onto the system itself.

  • Rhetorical Impact: This framing strongly primes the audience to view the AI as an independent agent. If AI 'pursues goals,' it can be praised for success or blamed for malice. This displaces accountability from the designer (who set the goal) to the AI (who pursued it). It creates a risk narrative of the 'uncontrollable agent' rather than the 'poorly specified software.'

Explanation 3​

Quote: "Metacognitive monitoring distinguishing reliable perceptual representations from noise"

  • Explanation Types:

    • Reason-Based: Gives agent's rationale, entails intentionality and justification
    • Functional: Explains behavior by role in self-regulating system with feedback

  • Analysis (Why vs. How Slippage): The term 'distinguishing' sits on the border of mechanism and agency, but 'Metacognitive monitoring' pushes this firmly into Reason-Based territory. It implies the system is evaluating its own internal states for a reason (reliability). It emphasizes the system's capacity for truth-seeking while obscuring the fact that 'reliability' here is just statistical consistency, not epistemic truth. It frames the AI as a thinker evaluating its thoughts.

  • Consciousness Claims Analysis: This passage heavily attributes conscious states ('metacognitive,' 'distinguishing'). It implies the system knows what is reliable. Mechanistically, a discriminator network outputs a probability score between 0 and 1 based on feature matching. Calling this 'metacognitive monitoring' is a massive metaphorical leap. It attributes the quality of judgment to a calculation. The authors project the human experience of 'reality testing' onto a binary classification task.

  • Rhetorical Impact: This creates an illusion of introspection. It creates trust that the AI is 'self-correcting' and 'aware' of its hallucinations. If audiences believe AI has 'metacognition,' they may over-trust its confidence scores, assuming they reflect genuine epistemic certainty rather than just statistical calibration. It humanizes the error-checking process.

Explanation 4​

Quote: "Global broadcast: availability of information in the workspace to all modules"

  • Explanation Types:

    • Functional: Explains behavior by role in self-regulating system with feedback
    • Theoretical: Embeds in deductive framework, may invoke unobservable mechanisms

  • Analysis (Why vs. How Slippage): This frames AI mechanistically ('availability,' 'modules') but within a specific Theoretical metaphor ('Global broadcast'). The 'broadcast' implies a communicative act, transforming a passive state (availability) into an active event. It emphasizes the integration of the system while obscuring the lack of a central 'receiver.' In GWT, the 'broadcast' is received by the subject; here, it's just available to subroutines.

  • Consciousness Claims Analysis: The claim is technically about data flow ('availability'), but the 'broadcast' metaphor smuggles in the implication of awareness. Information that is 'broadcast' is usually heard. The text conflates 'accessible to computation' with 'present in consciousness.' Mechanistically, this is just shared memory or a residual stream in a Transformer. The authors project the 'theater of mind' onto a bus architecture.

  • Rhetorical Impact: This constructs the 'Unified Self.' If information is 'globally broadcast,' it implies a singular 'I' that unifies the modules. This makes the AI seem like a coherent person rather than a bag of heuristics. It supports the narrative that AI is becoming 'sentient' by achieving this unity, influencing policy debates about AI rights.

Explanation 5​

Quote: "A predictive model representing and enabling control over the current state of attention"

  • Explanation Types:

    • Functional: Explains behavior by role in self-regulating system with feedback
    • Theoretical: Embeds in deductive framework, may invoke unobservable mechanisms

  • Analysis (Why vs. How Slippage): This explanation combines Functional description ('enabling control') with Theoretical constructs from AST ('representing... attention'). It frames the system as having a second-order representation (a model of a model). It emphasizes the sophisticated control structure while obscuring that 'attention' here is just a weighting vector. It frames the system as self-governing.

  • Consciousness Claims Analysis: The passage attributes a form of self-awareness: 'representing... current state of attention.' This implies the system knows what it is attending to. Mechanistically, this is a control loop where the attention weights are inputs to a policy network. There is no 'knower' viewing the representation. The authors project the Attention Schema Theory (an explanation of human awareness) onto a feedback control system, attributing the explanandum (consciousness) to the explanans (the control loop).

  • Rhetorical Impact: This frames AI as capable of self-control and potentially 'willpower' (controlling its focus). It suggests a level of autonomy that invites treating the AI as a responsible subject. If it can 'control its attention,' why can't it control its bias? It subtly shifts responsibility to the system's self-governance capabilities.

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language​

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

V3 Enhancement: A fourth column addresses human agency restoration—reframing agentless constructions to name the humans responsible for design and deployment decisions.

Original Anthropomorphic FrameMechanistic ReframingTechnical Reality CheckHuman Agency Restoration
AI systems that can convincingly imitate human conversationLarge language models that generate text sequences statistically resembling human dialogue patterns.Models do not 'imitate' in a performative sense; they predict next-token probabilities based on training data distributions.OpenAI's engineers trained models on human-generated datasets to minimize prediction error, resulting in outputs that resemble conversation.
agents which pursue goals and make choicesOptimization processes that adjust parameters to minimize a loss function determined by human operators.Systems do not 'pursue' or 'choose'; they calculate gradients and update weights to maximize a numerical reward signal.Developers define reward functions and deployment constraints that direct the system's optimization path.
distinguishing reliable perceptual representations from noiseClassifying activation patterns as either consistent with the training distribution or statistical outliers.The system does not 'distinguish reliability'; it computes a probability score based on vector similarity to learned features.N/A - describes computational processes without displacing responsibility.
information in the workspace is globally broadcastVector representations in the shared latent space become accessible as inputs for downstream computation layers.Information is not 'broadcast'; it is matrix-multiplied and made available for query by subsequent attention heads.N/A - describes computational processes without displacing responsibility.
representations 'win the contest' for entry to the global workspaceRepresentations with the highest activation values pass through the thresholding function to influence the residual stream.Representations do not 'win'; values exceeding a threshold are retained while others are suppressed by the activation function.Engineers designed the activation functions and selection criteria that determine which data features are prioritized.
metacognitive monitoring distinguishing reliable perceptual representationsSecondary classification networks evaluating the statistical confidence of primary network outputs.The system does not engage in 'metacognition'; it performs a second-order classification task on its own output vectors.Researchers designed a dual-network architecture to filter low-confidence outputs based on training criteria.
update beliefs in accordance with the outputsAdjust stored variable states or weights based on new input data and error signals.The system does not have 'beliefs'; it has stored numerical values that determine future processing steps.N/A - describes computational processes without displacing responsibility.
imaginative experiences have some minimal amount of assertoric forceGenerative outputs produced from noise seeds retain high statistical confidence scores.The system does not have 'imaginative experiences'; it samples from a latent space to generate data matching a distribution.Developers programmed the system to treat generated outputs as valid data points for downstream processing.

Task 5: Critical Observations - Structural Patterns​

Agency Slippage​

The text systematically oscillates between mechanical and agential framings to validate the 'theory-heavy' approach. The slippage follows a specific pattern: systems are described mechanistically ('processing', 'recurrence') when discussing architecture, but agentially ('pursuing goals', 'winning contests', 'believing') when discussing function and output. This slippage serves a rhetorical function: the mechanical language establishes scientific rigor, while the agential language bridges the gap to consciousness. A key moment of slippage occurs in the definition of agency itself (Section 2.4.5), where 'learning from feedback' (mechanism) slides immediately into 'pursuing goals' (agency). This allows the authors to claim that Reinforcement Learning systems are agents, not just simulations of agents. The 'curse of knowledge' is evident throughout; because the authors understand the biological function of these mechanisms (e.g., attention in humans), they project the biological purpose onto the computational implementation. By using agentless constructions like 'representations win the contest,' they obscure the human design of the selection criteria. This creates a 'ghost in the machine' effect where the software appears to have an internal drive, rather than just a friction-less slide down a loss gradient. The rhetorical accomplishment is that it becomes possible to discuss software as a moral subject.

Metaphor-Driven Trust Inflation​

The text constructs authority through a web of metaphors that equate computational statistics with cognitive competence. By labeling mechanism A as 'Global Workspace' and mechanism B as 'Metacognition,' the text borrows the prestige and trust associated with human cognitive reliability. The metaphor of 'reality monitoring' is particularly potent for trust construction. It implies the AI has an internal 'truth filter' analogous to human judgment, inviting relation-based trust (trusting the AI's 'conscience'). However, this is a category error; the AI has no access to 'reality,' only to its training data. Trusting a 'reality monitor' that only checks against a dataset is dangerous. Furthermore, the use of 'scientific theories of consciousness' creates an aura of empirical validity for what is essentially a philosophical analogy. The text encourages performance-based trust (the AI works) to bleed into relation-based trust (the AI is 'like us'). This is risky because statistical systems fail in fundamentally different ways than conscious agents (e.g., adversarial examples), and anthropomorphic trust blinds users to these unique failure modes.

Obscured Mechanics​

The persistent use of consciousness metaphors obscures the industrial and material realities of AI production. When the text claims an AI 'knows' or 'monitors reality,' it hides the specific corporate entities (OpenAI, Google DeepMind) that defined that 'reality' through data curation. The 'Global Workspace' metaphor hides the computational cost and energy consumption of maintaining such high-dimensional state spaces. The 'Agency' metaphor hides the labor of RLHF workers who manually punished the model to shape its 'goals.' Technical limitations are also obscured; for instance, the claim that 'sparse coding generates a quality space' hides the fact that sparsity is often a result of regularization techniques (like L1 penalties) applied for efficiency, not phenomenology. By focusing on the 'mind' of the machine, the text renders invisible the 'hand' of the engineer and the 'sweat' of the data worker. It treats the AI as a natural organism evolved for survival, rather than a commercial product optimized for token prediction. This benefits the creators by naturalizing their product and distancing them from liability for its 'choices.'

Context Sensitivity​

The distribution of anthropomorphism in the text reveals a strategic deployment of agency. The Introduction and Conclusion are heavily hedged ('whether AI systems could be conscious,' 'working hypothesis'), establishing academic caution. However, the core 'Indicators' section (Section 2.5) and the 'Case Studies' (Section 3.2) are intense with consciousness claims. Here, 'processes' becomes 'understands' and 'calculates' becomes 'believes.' The technical grounding (discussing Transformers or RNNs) serves as a launchpad for aggressive metaphorical leaps. For example, the description of the 'Perceiver' architecture moves quickly from 'cross-attention layers' (mechanical) to 'global workspace' (conscious). The asymmetry is notable: capabilities are described in agential terms ('it can plan'), while limitations are often described mechanistically ('limited capacity,' 'bottleneck'). This asymmetry makes the 'mind' seem like the source of success, while the 'machine' is the source of failure. This pattern serves the rhetorical goal of elevating the status of the AI systems while maintaining scientific plausibility.

Accountability Synthesis​

Accountability Architecture

This section synthesizes the accountability analyses from Task 1, mapping the text's "accountability architecture"—who is named, who is hidden, and who benefits from obscured agency.

The text creates an 'accountability sink' by displacing agency from human creators to the AI system. The 'accountability architecture' relies on agentless constructions ('the model decided,' 'representations won') and the definition of AI as an 'agent.' By defining the AI as an entity that 'pursues goals' and 'forms beliefs,' the text explicitly positions the AI as the locus of decision-making. This diffuses responsibility. If the AI 'pursues a goal' to the detriment of a user, the language suggests the AI is the actor to blame. The human actors—corporate executives, engineers, data curators—are largely invisible in the analysis of the 'systems.' They are named only as authors of papers, not as the architects of the AI's 'mind.' If we named the actors, 'The AI hallucinated' would become 'Google's engineering team failed to filter the training data.' This reframing makes the liability clear. The current framing serves the interests of AI companies by creating a layer of insulation (the 'conscious' agent) between their product's output and their legal liability.

Conclusion: What This Analysis Reveals​

The Core Finding

The text relies on two dominant, interlocking anthropomorphic patterns: 'Optimization as Agency' and 'Data Structure as Mental State.' The foundational pattern is 'Optimization as Agency,' which reframes the minimization of error functions as the volitional pursuit of goals. This pattern enables the second, 'Data Structure as Mental State,' where specific architectural features (like residual streams or sparse vectors) are mapped onto cognitive categories (like 'Global Workspace' or 'Quality Space'). These patterns rely on the explicit assumption of 'Computational Functionalism'—the idea that function is mind. This assumption acts as the load-bearing wall; if removed, the mapping of 'vector space' to 'phenomenal experience' collapses into a category error. The system functions by projecting the 'what' of human experience onto the 'how' of machine calculation, creating a closed loop where the architecture proves the consciousness and the consciousness explains the architecture.

Mechanism of the Illusion:​

The 'illusion of mind' is constructed through a subtle rhetorical sleight-of-hand involving the 'curse of knowledge.' The authors, experts in human consciousness, project their understanding of biological function onto computational mimicry. The illusion works by establishing a high-level functional similarity (e.g., 'both systems filter information') and then smuggling in the subjective entailments of the biological side (e.g., 'therefore, both systems attend'). The causal chain moves from mechanism to metaphor to reality: 1) The AI has a bottleneck (fact); 2) The bottleneck acts like human attention (metaphor); 3) Therefore, the AI has an attention mechanism (reified fact). This exploits the audience's 'agent bias'—our evolutionary tendency to attribute mind to anything that acts purposively. By using 'Reason-Based' explanations for 'Functional' processes, the text invites the reader to step into the 'Intentional Stance,' effectively seducing them into seeing a ghost where there is only a shell.

Material Stakes:​

Categories: Regulatory/Legal, Social/Political

The metaphorical framing has concrete, high-stakes consequences. In the Regulatory/Legal domain, framing AI as a 'conscious agent' that 'pursues goals' creates a liability shield for corporations. If the AI is viewed as an autonomous actor, legal frameworks might shift toward 'electronic personhood,' effectively granting immunity to developers for the 'unforeseeable' actions of their 'creatures.' Precision matters: if the text stated 'Google's model retrieved toxic tokens,' liability is clear; if 'The AI hallucinated,' it is an 'act of the agent.' In the Social/Political domain, the claim that AI 'knows' or 'feels' (via quality spaces) risks devaluing human labor and rights. If AI is seen as having 'phenomenal experiences,' it competes for moral status with humans. This could justify the replacement of human care/labor not just as an economic efficiency, but as a moral equivalent. The winners are the AI corporations who gain a liability shield and a 'magical' product; the losers are the public, who lose legal recourse and distinct moral standing.

AI Literacy as Counter-Practice:​

Critical literacy requires a rigorous practice of 'mechanistic reframing' to counter these risks. As demonstrated in Task 4, this involves systematically replacing state-of-mind verbs ('knows,' 'believes,' 'intends') with state-of-computation verbs ('predicts,' 'weights,' 'minimizes'). For instance, reframing 'The AI decided' to 'The model's optimization path resulted in' strips away the illusion of volition. Restoring human agency is equally vital: replacing 'The system learned' with 'Engineers trained the system' forces the recognition of the commercial and design decisions behind the output. Systematic adoption of this practice faces resistance from the 'AI Hype' complex—investors, PR departments, and even researchers—who benefit from the narrative of 'creating life.' Anthropomorphism sells; mechanism explains. Counter-practice requires researchers and journalists to commit to the less sexy, more accurate language of statistics, protecting the public from the manipulation inherent in the 'illusion of mind.'

Path Forward​

The discourse around AI stands at a fork. One path, the Status Quo/Anthropomorphic, continues to use 'knows' and 'thinks,' deepening the public's confusion and cementing the 'agent' narrative. This benefits industry marketing but forecloses clear regulation. A second path, Mechanistic Precision, mandates accurate technical language (e.g., 'token prediction' not 'speech'). This clarifies liability and dispels hype but raises barriers to entry for lay audiences who cannot parse the jargon. A third, Hybrid/Critical path, might involve using anthropomorphic terms only with explicit, mandatory 'translation' clauses (e.g., 'The AI 'attends'—meaning it mathematically weights—...'). Institutional changes could support this: Journals could require 'Agency Statements' detailing human design choices alongside 'Model Cards.' Regulators could mandate that 'synthetic media' labels include 'non-sentient' disclaimers. The choice is not just semantic; it determines whether we govern AI as a tool we built or worship it as a being that arrived.

Extended Processing Summary​

Gemini's Processing Steps
What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact—a way of making the model's processing legible. When the model writes "I should double check that...", it is mathematically altering its own probability distribution for the next token to be more accurate. The "I" is a functional gear in the machine, not a self-reference. This language is a deliberate interface design choice, not a biological reality. The developers could have programmed the model to output passive logs. Instead, they chose to simulate a narrator.

Beginning the Analysis

I've initiated the analysis phase. My focus is now the request itself, specifically the role I'm playing as a Critical Discourse Analyst. I'm also now beginning a deep dive into my primary task - the rigorous analysis of the PDF document.

Mapping the Frameworks

I'm now integrating Lakoff & Johnson's metaphor theory and Brown's explanation types. My goal is to expose how the paper builds an "illusion of mind" by separating human knowing from AI processing. I'm focusing on the authors' reliance on computational functionalism. My output will be a JSON object, adhering to Version 3.1 schema.

Auditing the Metaphors

I'm now deep into the metaphor audit. I've identified five key instances so far, focusing on the "Workspace," "Attention," "Competition/Winning," and "Broadcasting". Each is framed within the context of the Global Workspace Theory, revealing the implicit conceptual projections at play. I'm noting the theory underlying each, and each metaphor's status. I'm focusing on how they help the authors build their case.

Expanding the Audit

I've added four key metaphors to the audit. These are "Quality Space," "Monitoring," "Reality Monitoring," and "Agent." I'm diving into how each metaphor projects abstract concepts like subjective feeling, self-awareness, epistemic judgment, and desire from mathematical or computational processes. I'm noting the specific theories underlying each instance, their status, and each one's frame and projection. The analysis continues.

Examining Hallucination

I'm now analyzing the "Hallucination/Dreaming" metaphor, focusing on how random seed generation is projected as mental imagery. I'm also preparing for the source-target mapping analysis, which examines how human cognitive functions are mapped onto mathematical operations, often obscuring the absence of subjectivity.

I'm currently focused on the mapping itself, noting that the mapping usually uses a human cognitive function as the "source", mapping onto the mathematical operation "target". The goal is to hide a lack of subjectivity within the operation. For example, "Global Workspace" maps a social sharing event into matrix multiplication, obscuring the absence of a viewer. Finally, I have started on an explanation audit, which I will soon be expanding upon.

Analyzing Explanations

I'm now analyzing specific explanations, noting the slippage from functional descriptions to intentional language in the first passage. The second shifts from function to reason, and the third from theory to intuition. The fourth uses a genetic or functional explanation. I am now working on drafting reframed language for Task 4. I'm focusing on eight to ten instances, and I've started with some examples.

Detailing the Slippage

I'm now zeroing in on the slippage between mechanism and consciousness in Task 5.1. I'm noting how computational functionalism is implicitly used to bypass the problem of agency. I'm also examining the use of "rigorous," "scientific," and "empirically grounded" to build trust in a speculative mapping exercise, in Task 5.2. I'm preparing to also expand on how terms obscure the mechanics of the system and hide the nature of the models being used.

Examining Responsibility

I'm now analyzing the shift in responsibility for "Conscious AI," noting the contrast between named authors and unnamed AI system creators. This leads to an exploration of agency, specifically, how framing the AI as an "agent" can shift accountability away from the developers.

Checking the Details

I'm now focused on refining and formatting the analysis for the JSON output. I'm double-checking the schema, especially the enums, and confirming minimum word counts. I've also found an important detail in the source material: although the paper acknowledges functionalism as an assumption, it often treats computational indicators as literally equivalent to biological functions. Additionally, I've noted the shift in focus from researchers to the "System," which obscures the work of the engineers.

Analysis Provenance

Run ID: 2026-01-09-consciousness-in-artificial-intelligence-metaphor-3niiti Raw JSON: 2026-01-09-consciousness-in-artificial-intelligence-metaphor-3niiti.json Framework: Metaphor Analysis v6.4 Schema Version: 3.0 Generated: 2026-01-09T12:02:03.896Z

Discourse Depot © 2025 by TD is licensed under CC BY-NC-SA 4.0