Skip to main content

Financial Stability Implications of Generative AI - Taming the Animal Spirits

About

This document presents a Critical Discourse Analysis focused on AI literacy, specifically targeting the role of metaphor and anthropomorphism in shaping public and professional understanding of generative AI. The analysis is guided by a prompt that draws from cognitive linguistics (metaphor structure-mapping) and the philosophy of social science (Robert Brown's typology of explanation).

All findings and summaries below were generated from detailed system instructions provided to a large language model and should be read critically as interpretive outputs—not guarantees of factual accuracy or authorial intent.

Task 1: Metaphor and Anthropomorphism Audit​

About this task

For each of the major metaphorical patterns identified, this audit examines the specific language used, the frame through which the AI is being conceptualized, what human qualities are being projected onto the system, whether the metaphor is explicitly acknowledged or presented as direct description, and—most critically—what implications this framing has for trust, understanding, and policy perception.

1. Software as Economic Agent​

Quote: "Our results show that AI agents make more rational decisions than humans"

  • Frame: Model as a Thinking Organism/Economic Actor
  • Projection: This metaphor maps the human capacity for autonomous, intentional decision-making and rational judgment onto a statistical text generation process. By labeling the software an 'agent' that makes 'rational decisions,' the text projects the quality of conscious deliberation—the ability to weigh evidence, understand consequences, and form justified beliefs—onto an algorithm. It implies the system 'knows' the economic context and 'understands' the concept of rationality, rather than simply calculating token probabilities that statistically mirror rational behaviors found in its training data.
  • Acknowledgment: Direct description (some hedging in Introduction, but 'AI agents' is used as a literal term throughout).
  • Implications: This framing dangerously inflates the perceived sophistication of the system by suggesting it possesses the cognitive architecture necessary for rationality (justified true belief). If policymakers and investors believe AI systems are 'rational agents' rather than probabilistic tools, they may assign them levels of authority and autonomy that are unwarranted. It creates a risk of over-trusting algorithmic outputs in financial markets, masking the reality that these systems can fail in non-rational, mechanistic ways (e.g., hallucination, adversarial attacks) that human 'irrationality' does not mimic.
Show more...

2. Inheritance of Biological Conditioning​

Quote: "AI agents are not purely algorithmic, but have inherited some elements of human conditioning and bias."

  • Frame: Cognition as a Biological/Evolutionary Process
  • Projection: This metaphor projects the biological and psychological concept of 'inheritance'—the transmission of traits via genetics or cultural upbringing—onto the data training process. It implies the AI 'possesses' these traits as internal psychological states (conditioning/bias) that it 'knows' or 'experiences.' In reality, the model has simply optimized weights to minimize loss against a dataset containing these patterns. It suggests a conscious acquisition of human flaws rather than a mechanical reproduction of statistical artifacts.
  • Acknowledgment: Direct description.
  • Implications: Framing statistical artifacts as 'inherited conditioning' humanizes the error modes of the system. It suggests the AI's failures are 'human-like' (and therefore understandable or empathetic) rather than 'machine-like' (brittle, non-causal correlations). This risks creating a false sense of familiarity with the risks; a 'biased' human can be reasoned with or educated, whereas a model with statistical skew requires re-engineering or re-training. It obscures the technical reality of dataset curation in favor of a psychological narrative.

3. The Reluctant Mind​

Quote: "they still remain reluctant to herding."

  • Frame: Psychological Disposition/Emotion
  • Projection: This creates a strong projection of conscious intent and emotional resistance ('reluctant'). It implies the AI 'knows' what herding is and 'chooses' to resist it due to an internal preference or psychological state. Resistance requires awareness of an impulse and a will to counter it. The metaphor attributes this subjective inner life to the system, masking the mechanistic reality that the probability distribution simply favored contrarian or private-signal tokens over consensus tokens.
  • Acknowledgment: Direct description.
  • Implications: Attributing 'reluctance' to a software program suggests it has a personality or moral compass. This anthropomorphism encourages users to treat the system as a partner with agency ('it didn't want to herd') rather than a tool. In a financial stability context, this is dangerous because it implies the system will consistently 'behave' a certain way due to character, rather than outputting tokens based on specific, potentially fragile, prompt conditions and temperature settings.

4. The Conscious Reasoner​

Quote: "Studying the reasoning of the LLMs for these runs reveals that the models do not take their profiles into account"

  • Frame: Text Generation as Cognitive Reasoning
  • Projection: This metaphor maps the human cognitive process of introspection and justification ('reasoning') onto the chain-of-thought text generation capabilities of the model. It claims the AI 'reasons'—a verb implying conscious logical deduction and understanding of causality. It suggests the AI 'knows' why it made a decision. Mechanistically, the 'reasoning' is just more generated text, produced by the same predictive probabilities as the decision itself, not a window into a 'mind' that made the choice.
  • Acknowledgment: Direct description.
  • Implications: This is perhaps the most deceptive metaphor in the text. By treating generated text as 'reasoning,' the authors validate the illusion that the AI has a coherent internal thought process. This risks the 'curse of knowledge' where humans read meaning into the output that isn't there. If financial regulators believe AI 'reasons,' they may audit its 'explanations' rather than its code and training data, leading to a superficial and dangerous form of oversight.

5. The Unaware Agent​

Quote: "AI agents fail to acknowledge this, leading to occasional suboptimal choices."

  • Frame: Cognition as Awareness
  • Projection: The phrase 'fail to acknowledge' implies a lapse in conscious attention or awareness. It suggests the AI 'could have known' or 'should have understood' the context but missed it, like a distracted human. It projects a capacity for 'knowing' context that was momentarily unused. Mechanistically, the model simply failed to attend to specific tokens in the context window or lacked the requisite patterns in its weights to correlate history with optimal outcome.
  • Acknowledgment: Direct description.
  • Implications: This framing obscures the technical limitations of the attention mechanism. It implies the system has a general capacity for understanding that simply glitched, rather than a fundamental limitation in how it processes sequential information. This can lead to overestimation of the system's reliability, assuming that with better 'focus' (prompting) it will 'understand,' when in reality it may lack the representational capacity for the specific task.

6. The Guided Student​

Quote: "AI agents can be induced to herd optimally when explicitly guided"

  • Frame: Model Tuning as Educational Guidance
  • Projection: This metaphor maps the teacher-student relationship onto prompt engineering. 'Guided' implies a transfer of understanding where the student grasps a concept. It suggests the AI 'understands' the guidance and modifies its behavior based on that new knowledge. In reality, the prompt provides additional context tokens that shift the conditional probability distribution of the output. The AI doesn't 'learn' or 'follow guidance'; it completes a pattern based on new input constraints.
  • Acknowledgment: Direct description.
  • Implications: Framing prompt engineering as 'guidance' anthropomorphizes the control interface. It suggests a collaborative social relationship between user and tool. This can lead to 'prompt superstition,' where users believe they can 'talk' the AI into better performance through rhetoric rather than precise technical instruction. In financial contexts, expecting an AI to follow 'guidance' like a human employee creates liability ambiguity when the system inevitably fails to 'understand' the nuance of an instruction.

7. Perception of Color​

Quote: "the models perceive white and blue as neutral signals"

  • Frame: Data Processing as Sensory Perception
  • Projection: The verb 'perceive' attributes subjective sensory experience and interpretation to the system. It suggests the AI 'sees' the colors and 'interprets' their meaning through a cognitive lens. Mechanistically, the model processes the token strings 'white' and 'blue' and associates them with 'neutral' based on vector embeddings from training data. There is no perception, only token correlation.
  • Acknowledgment: Direct description.
  • Implications: Claiming the AI 'perceives' signals obscures the fragility of token processing. If the system 'perceives,' we expect it to be robust to minor variations. However, since it only processes tokens, slight changes in phrasing (not color) could radically alter the output. This metaphor constructs an illusion of a sensory agent interacting with the world, rather than a text processor manipulating symbols.

Task 2: Source-Target Mapping​

About this task

For each key metaphor identified in Task 1, this section provides a detailed structure-mapping analysis. The goal is to examine how the relational structure of a familiar "source domain" (the concrete concept we understand) is projected onto a less familiar "target domain" (the AI system). By restating each quote and analyzing the mapping carefully, we can see precisely what assumptions the metaphor invites and what it conceals.

Mapping 1: Economic Theory / Human Psychology → Statistical Output Generation​

Quote: "AI agents make more rational decisions than humans"

  • Source Domain: Economic Theory / Human Psychology
  • Target Domain: Statistical Output Generation
  • Mapping: The structure of a 'rational agent'—a hypothetical human who maximizes utility based on available information—is mapped onto the output of a Large Language Model. This implies the LLM possesses 'beliefs' about the state of the world and makes 'choices' to maximize a reward function based on those beliefs. It projects the conscious state of 'rationality' (weighing reasons) onto the mechanistic state of 'low perplexity' (predicting high-probability tokens).
  • What Is Concealed: This mapping conceals the absence of intentionality and utility-maximization in the model itself. The LLM is not trying to make money or maximize utility; it is minimizing the cross-entropy loss of the next token. It hides the fact that the 'rationality' is a simulation—a statistical mimicry of rational texts found in the training data—rather than an internal property of the system.
Show more...

Mapping 2: Biology / Cultural Transmission → Training Data Distribution​

Quote: "inherited some elements of human conditioning and bias"

  • Source Domain: Biology / Cultural Transmission
  • Target Domain: Training Data Distribution
  • Mapping: The source domain involves the transmission of traits from parent to child or culture to individual, implying an internal acquisition of characteristics. This is mapped onto the training phase of an LLM. It suggests the model 'has' these biases in the same way a human has a personality trait or a conditioned reflex. It implies a psychological depth where these biases reside.
  • What Is Concealed: This conceals the mechanistic reality of data ingestion. The 'bias' is not a psychological conditioning but a statistical skew in the corpus. The model does not 'have' a bias; it retrieves tokens from a biased distribution. This mapping hides the material labor of data curation and the specific composition of the datasets (e.g., Reddit, Twitter) mentioned in the text.

Mapping 3: Conscious Judgment / Evaluation → Probabilistic Token Ranking​

Quote: "AI agents generally deem that there is an opportunity"

  • Source Domain: Conscious Judgment / Evaluation
  • Target Domain: Probabilistic Token Ranking
  • Mapping: The source domain is the human act of 'deeming'—a conscious judgment call based on evaluation of evidence. This is mapped onto the model's generation of text indicating a favorable trade. It implies the AI 'surveys' the situation, 'evaluates' the risk, and 'forms an opinion.'
  • What Is Concealed: This conceals the lack of a world model. The AI cannot 'deem' anything because it has no concept of 'opportunity' or 'value.' It conceals the process of vector similarity where the context tokens (price, signal) statistically correlate with positive sentiment tokens ('attractive', 'buy') in the model's high-dimensional space.

Mapping 4: Cognitive Psychology / Introspection → Chain-of-Thought Text Generation​

Quote: "Studying the reasoning of the LLMs"

  • Source Domain: Cognitive Psychology / Introspection
  • Target Domain: Chain-of-Thought Text Generation
  • Mapping: The source domain is the study of human thought processes—why someone did what they did. This is mapped onto the analysis of auxiliary text generated by the model. It assumes the text caused the decision or accurately reflects the internal state that led to the decision, just as human reasoning reflects human intent.
  • What Is Concealed: This conceals the confabulation inherent in LLMs. The 'reasoning' is generated after or alongside the decision using the same predictive mechanism; it is not the cause of the decision. The model effectively hallucinates a rationale that fits the decision token. This mapping hides the fact that there is no causal link between the 'reasoning' text and the 'action' token other than statistical adjacency.

Mapping 5: Emotional State / Will → Probability Distribution​

Quote: "reluctant to herding"

  • Source Domain: Emotional State / Will
  • Target Domain: Probability Distribution
  • Mapping: The source domain is human emotional resistance or force of will. This is mapped onto a statistical tendency to avoid repeating the majority action token. It implies an internal struggle or a preference structure that resists peer pressure.
  • What Is Concealed: This conceals the mathematical nature of the 'reluctance.' The model isn't 'holding back'; the weights simply assign a lower probability to the 'buy' token when the history shows 'buy, buy, buy.' It hides the absence of social pressure in the machine; there is no 'herd' for the AI, only a sequence of input tokens.

Mapping 6: Pedagogy / Mentorship → Prompt Engineering / Context Injection​

Quote: "explicitly guided to make profit-maximizing decisions"

  • Source Domain: Pedagogy / Mentorship
  • Target Domain: Prompt Engineering / Context Injection
  • Mapping: The source domain is a teacher guiding a student to a better understanding. This is mapped onto the user adding specific constraints to the input prompt. It implies the AI 'understands' the goal of profit maximization once pointed out.
  • What Is Concealed: This conceals the prompt-dependency and brittleness of the system. The 'guidance' is just modifying the input vector to shift the output probability surface. It hides the fact that the AI has no concept of 'profit' or 'maximization,' only of token patterns that follow 'optimize for X' instructions in the training data.

Mapping 7: Psychological Vulnerability → Algorithmic Determinism​

Quote: "susceptible to preconceived biases"

  • Source Domain: Psychological Vulnerability
  • Target Domain: Algorithmic Determinism
  • Mapping: The source domain is a human mind that can be influenced or prejudiced. This is mapped onto the model's tendency to output stereotypical or erroneous responses when prompted with specific triggers (like colors). It implies a cognitive weakness or flaw in judgment.
  • What Is Concealed: This conceals the rigid determinism of the model. The 'susceptibility' is actually a high-fidelity reproduction of training data correlations. If red is usually 'bad' in the training data, the model predicts 'bad.' It obscures the fact that the 'bias' is a feature of the model's success in pattern matching, not a psychological failure of judgment.

Task 3: Explanation Audit (The Rhetorical Framing of "Why" vs. "How")​

About this task

This section audits the text's explanatory strategy, focusing on a critical distinction: the slippage between "how" and "why." Based on Robert Brown's typology of explanation, this analysis identifies whether the text explains AI mechanistically (a functional "how it works") or agentially (an intentional "why it wants something"). The core of this task is to expose how this "illusion of mind" is constructed by the rhetorical framing of the explanation itself, and what impact this has on the audience's perception of AI agency.

Explanation 1​

Quote: "Why do LLMs engage in contrarian behavior although theory predicts that this type of decision is never optimal? One explanation is that AI agents fail to incorporate trading history into their expectations of the asset's value."

  • Explanation Types:

    • Intentional: Refers to goals or purposes and presupposes deliberate design, used when the purpose of an act is puzzling
    • Functional: Explains a behavior by its role in a self-regulating system that persists via feedback, independent of conscious design

  • Analysis (Why vs. How Slippage): The authors frame the 'why' using intentional language ('engage in', 'fail to incorporate'). They attribute the behavior to a lapse in the agent's cognitive process—specifically, a failure to perform a mental action ('incorporate history'). This emphasizes a psychological deficit in a 'rational agent' rather than a mechanical limitation. A mechanistic explanation would focus on the attention mechanism's inability to attend to the long-context history tokens or the lack of training examples where history overrides private signal. By framing it as a 'failure to incorporate,' they imply the agent could have done so but 'missed' it, obscuring the algorithmic constraint.

  • Consciousness Claims Analysis: The passage uses consciousness-implying phrasing like 'expectations of the asset's value' and 'fail to incorporate.' To 'expect' a value is to hold a belief about a future state; to 'incorporate' history is to consciously integrate evidence into that belief. The text treats the AI as a 'knower' that should possess a justified belief about the asset price but failed to form it correctly. This projects the authors' understanding of the optimal trading strategy (Bayesian updating) onto the AI, assuming the AI 'wants' to value the asset correctly but failed. The 'curse of knowledge' is evident: the authors know the trading history matters, so they describe the AI's behavior as a failure to know what they know. The actual mechanistic process is that the model's attention weights for the 'history' tokens were insufficient to override the probability distribution generated by the 'private signal' tokens. The model does not have 'expectations'; it has conditional probability distributions.

  • Rhetorical Impact: Framing this as a cognitive 'failure' rather than a technical limitation suggests that the AI is close to human-like rationality, just with some 'blind spots.' This humanizes the error. It suggests that with better 'training' or 'focus' (prompting), the AI could 'learn' to incorporate the history. This inflates the perceived agency of the system, making it seem like a junior trader who needs mentoring, rather than a statistical tool that requires code alteration. It invites trust in the system's general competence, positioning the error as an exception rather than a feature of its architecture.

Show more...

Explanation 2​

Quote: "AI agents generally deem that there is an opportunity to make reasonable profits by engaging in trading."

  • Explanation Types:

    • Reason-Based: Gives the agent's rationale or argument for acting, which entails intentionality and extends it by specifying justification

  • Analysis (Why vs. How Slippage): This is a quintessential reason-based explanation. It explains the 'act' (trading) by citing the agent's 'rationale' (deeming there is an opportunity). This completely agentializes the system, treating it as a strategic actor making choices based on evaluated incentives. It obscures the mechanistic reality that the model is simply outputting 'buy' or 'sell' tokens because those tokens are statistically most probable following the prompt structure, not because it has evaluated an 'opportunity.'

  • Consciousness Claims Analysis: The verb 'deem' is a strong consciousness verb, implying judgment, evaluation, and the formation of a subjective opinion. The text explicitly claims the AI 'knows' or 'believes' there is an opportunity ('deem that there is...'). It treats the AI's output as the result of a conscious deliberation about profit ('make reasonable profits'). The 'curse of knowledge' is severe here: the authors interpret the output token (e.g., 'Buy') through their own economic framework, attributing the intent of profit-seeking to the machine. They conflate the output of a trade with the mental state of seeking profit. Mechanistically, the model calculates the dot product of key and query vectors in the transformer layers to predict the next token. It does not 'deem' anything; it converts input matrices into output probabilities. There is no concept of 'profit' or 'opportunity' within the floating-point operations.

  • Rhetorical Impact: This framing constructs the AI as a competent, economically rational actor. By saying it 'deems' an opportunity exists, the authors grant the AI authority—it has assessed the situation and made a judgment. This builds massive trust in the system's outputs. Investors reading this might believe the AI has 'insight' or 'intuition' about markets. It hides the risk that the AI might trade 'rationally' based on hallucinated data or flawed logic, because 'deeming' implies a validity to the judgment process that 'predicting' does not.

Explanation 3​

Quote: "If the model had taken into account the fact that the majority action in the trading history was to sell the asset, it may have assigned a higher probability of a low asset value..."

  • Explanation Types:

    • Intentional: Refers to goals or purposes and presupposes deliberate design, used when the purpose of an act is puzzling
    • Theoretical: Embeds behavior in a deductive or model-based framework, may invoke unobservable mechanisms such as latent variables or attention dynamics

  • Analysis (Why vs. How Slippage): This passage blends a theoretical explanation ('assigned a higher probability') with intentional framing ('taken into account'). It anthropomorphizes the probabilistic mechanism. While 'assigned a higher probability' is technically accurate, the condition 'if the model had taken into account' implies a cognitive act of attention and consideration. It suggests the model has the agency to choose what to consider. It obscures the fact that 'taking into account' is purely a function of the attention mechanism's pre-calculated weights, not a run-time decision by the model.

  • Consciousness Claims Analysis: The phrase 'taken into account' implies a conscious awareness of data points and their semantic implications. The text treats the AI as a 'knower' that could have known the significance of the trading history but didn't. This is a subtle projection of consciousness—the ability to focus awareness. Mechanistically, 'taking into account' simply means the attention head values for the history tokens would need to be higher relative to the private signal tokens during the computation of the context vector. The text obscures this mathematical reality with the language of cognitive deliberation. It conflates the processing of tokens with the knowing of facts.

  • Rhetorical Impact: This framing implies that the AI's errors are lapses in judgment rather than structural limitations. It suggests a 'reasonableness' to the AI's failure—it just didn't 'notice' the history. This makes the system seem more human and relatable, potentially increasing trust (or at least forgiveness) for its errors. It minimizes the perceived risk of structural blindness in AI systems, where they cannot see certain correlations, by framing it as a momentary lapse of attention.

Explanation 4​

Quote: "The model appears to use more 'judgment' or 'emotion' in its reasoning... Llama 3 does not reason using the expected value conditional on the signal alone, but includes trading history..."

  • Explanation Types:

    • Dispositional: Attributes tendencies or habits such as inclined or tends to, subsumes actions under propensities rather than momentary intentions
    • Reason-Based: Gives the agent's rationale or argument for acting, which entails intentionality and extends it by specifying justification

  • Analysis (Why vs. How Slippage): This explanation attributes a personality trait ('judgment', 'emotion') to the model. It frames the difference in output variance (temperature effects or training data variance) as a difference in character or cognitive style. It suggests Llama 3 'prefers' or 'decides' to use history, while others don't. This obscures the mechanistic explanation: Llama 3's training data likely contained more narrative or context-dependent examples, leading its weights to favor history tokens more heavily than the Claude models.

  • Consciousness Claims Analysis: The claim that a model uses 'judgment' or 'emotion' is a direct projection of conscious subjective experience. 'Judgment' implies the ability to weigh values and make a wise choice; 'emotion' implies affective states. The text explicitly attributes 'reasoning' to the model. It treats the text generation process as a window into a 'mind' that is reasoning. The 'curse of knowledge' appears where the author reads the content of the generated text (which might use emotional words) and attributes that emotion to the system itself. Mechanistically, the model is simply sampling from a probability distribution with higher entropy or different attention patterns. There is no emotion, only token variance.

  • Rhetorical Impact: Attributing 'judgment' and 'emotion' to an AI is highly misleading. It suggests the AI has a 'soul' or a 'personality.' This fosters parasocial relationships and unjustified trust. Users might believe the AI 'feels' the market or has 'intuition,' leading them to rely on it for decisions that require genuine human insight. It effectively mystifies the technology, turning it into an oracle rather than a calculator.

Explanation 5​

Quote: "AI agents are thus 'averse to herding.'"

  • Explanation Types:

    • Dispositional: Attributes tendencies or habits such as inclined or tends to, subsumes actions under propensities rather than momentary intentions

  • Analysis (Why vs. How Slippage): This is a dispositional explanation, attributing a stable character trait ('averse') to the system. 'Averse' implies a psychological distaste or avoidance. It frames the statistical infrequency of herding behavior as a preference or personality quirk of the agent. This obscures the functional explanation: the objective function and training data simply did not reinforce copying behavior in this specific context.

  • Consciousness Claims Analysis: To be 'averse' is to have a subjective preference or a feeling of repulsion. This attributes a conscious emotional state to the system. It implies the AI 'knows' what herding is and 'dislikes' it. Mechanistically, the model simply generates 'buy' tokens (private signal) more often than 'sell' tokens (consensus) because of how the prompt interacts with its weights. There is no aversion, only probability. The text projects the authors' observation of the outcome (less herding) into a mental state of the agent (aversion).

  • Rhetorical Impact: Describing AI as 'averse to herding' makes it sound like a contrarian investor—a sophisticated, independent thinker. This frames the AI as a 'maverick' or a 'disciplined' trader, qualities that are highly engaged in finance. This rhetorical move acts as a powerful trust signal: 'this AI won't just follow the crowd; it has integrity.' It dangerously masks the fact that the 'aversion' is just a hard-coded statistical pattern that could fail catastrophically in a different context.

Task 4: AI Literacy in Practice - Reframing Anthropomorphic Language​

About this task

This section proposes alternative language for key anthropomorphic phrases, offering more mechanistic and precise framings that better reflect the actual computational processes involved. Each reframing attempts to strip away the projections of intention, consciousness, or agency that are embedded in the original language.

Original Anthropomorphic FrameMechanistic ReframingTechnical Reality Check
AI agents make more rational decisions than humansThe models generate trading outputs that align more frequently with theoretical profit-maximizing parameters than human participants do.The AI does not 'make rational decisions' or possess rationality. The model classifies the input scenario and generates a token response ('buy' or 'sell') based on probability distributions learned from training data that correlate with logic-puzzle solutions.
AI agents fail to acknowledge this, leading to occasional suboptimal choices.The model's attention mechanism did not sufficiently weight the trading history tokens relative to the private signal tokens, resulting in output probabilities that did not align with the optimal Bayesian update.The AI cannot 'acknowledge' or 'fail to acknowledge' facts, as it lacks conscious awareness. The system processes inputs via matrix multiplication; 'failure' here refers to the weights not producing the target token, not a lapse in cognitive attention.
inherited some elements of human conditioning and biasThe models reproduce statistical patterns of error and skew present in their training corpora.The AI does not 'inherit' conditioning or possess 'bias' as a psychological trait. It retrieves and recombines patterns from its dataset. If the data contains biased human text, the model predicts similar text sequences via statistical correlation.
they still remain reluctant to herding.The models consistently generated tokens corresponding to private signals rather than consensus actions, even when prompted with optimal strategy instructions.The AI has no emotional state of 'reluctance.' The term describes a statistical resistance where the probability of generating a 'follow the crowd' token remained low despite changes in the prompt.
AI agents generally deem that there is an opportunity to make reasonable profitsThe models generated text indicating a positive trade evaluation ('opportunity') based on the input parameters.The AI does not 'deem' or evaluate opportunities. It predicts that the sequence 'there is an opportunity' is a probable completion to the provided financial context prompt based on training examples.
explicitly guided to make profit-maximizing decisionsprompted with instructions describing the optimal profit-maximizing strategyThe AI is not 'guided' in an educational sense and does not make 'decisions.' The prompt inserts additional tokens into the context window, which alters the conditional probabilities of the subsequent generated tokens.
The model appears to use more 'judgment' or 'emotion' in its reasoningThe model generates text containing more affective and evaluative vocabulary in its chain-of-thought outputs.The AI does not possess 'judgment' or 'emotion.' Variations in output reflect differences in training data composition (e.g., more emotive texts) and sampling parameters (temperature), not cognitive or emotional states.
the models perceive white and blue as neutral signalsthe models associate the tokens 'white' and 'blue' with neutral sentiment vocabularyThe AI does not 'perceive' colors or signals. It processes text tokens. The vector embeddings for 'white' and 'blue' in this context do not trigger high-probability associations with 'bullish' or 'bearish' vocabulary.

Task 5: Critical Observations - Structural Patterns​

Agency Slippage​

The text exhibits a persistent and strategic oscillation between mechanical and agential framings, functioning to validate the 'AI as Agent' narrative while retaining scientific veneer. The slippage predominantly moves from mechanical setup to agential interpretation. In the methodology section (Section 5), the authors use relatively grounded language: 'we prompt the LLMs,' 'we make an API call,' 'temperature of 0.7.' However, as the text moves into the Results and Discussion (Sections 6-8), the language shifts dramatically to consciousness-attributing agency: 'AI agents fail to acknowledge,' 'remain reluctant,' 'deem that there is an opportunity.'

A critical moment of slippage occurs in the 'Reasoning' analysis (Section 6.4). The authors mechanically extract text generated by the model (a computational output), but then immediately treat this text as a valid psychological account of why the agent acted ('Analyzing these reasoning paragraphs sheds light on the decision making process'). They use the model's output (Pattern A) to explain the model's output (Pattern B), projecting a 'knower' behind the text who holds beliefs ('I believe the asset is...'). This 'curse of knowledge' mechanism is foundational: the authors know what human reasoning looks like, so they attribute the faculty of reasoning to the system when it mimics the form of reasoning. The slippage makes the 'illusion of mind' rhetorically load-bearing; the paper's conclusions about 'financial stability' rely on the premise that these are 'agents' with 'behaviors' (like reluctance or rationality) that can be generalized, rather than software functions that are brittle and prompt-dependent.

Metaphor-Driven Trust Inflation​

Trust is constructed in this text through the metaphor of the 'Rational Economic Actor.' By consistently applying the frame of 'rationality' (a highly positive value in economics) to the AI's performance, the text builds a powerful structure of authority. The metaphors distinguish between 'performance-based trust' (the AI made money) and 'relation-based trust' (the AI is 'reluctant,' 'judicious,' or 'rational'). The text explicitly encourages the latter. Phrases like 'AI agents make more rational decisions' and 'AI agents generally deem there is an opportunity' signal that the system is not just a tool, but a competent subject—a 'Professional Trader' (as one prompt explicitly names it).

Consciousness language plays a crucial role here. Claiming the AI 'knows' or 'reasons' implies that its decisions are justified, not just statistically probable. A random number generator can be 'lucky,' but only a rational agent can be 'correct.' By framing the AI as a 'knower' that 'avoids herding' due to 'reluctance' (a character trait), the text suggests the AI has integrity. This creates a dangerous form of trust where users (and regulators) might rely on the AI's 'judgment' in volatile markets, assuming it possesses a stability of character that a statistical model fundamentally lacks. The risk is that audiences will extend relation-based trust—expecting the AI to 'behave' responsibly—to a system that only processes tokens, leading to catastrophic misalignment when the statistical context shifts.

Obscured Mechanics​

The anthropomorphic system effectively obscures the material and technical realities of Large Language Models. Most notably, it hides the absence of a world model and the dependency on training data distribution. When the text claims the AI 'inherits bias' or 'perceives signals,' it masks the fact that the model is simply a mirror of its training corpus. The 'bias' isn't a psychological flaw to be therapized; it's a data sampling artifact. Similarly, the concept of 'rationality' obscures the fact that the AI has no concept of value, money, or time; it effectively 'hallucinates' the rational move because 'rational' texts dominate its fine-tuning data on economic tasks.

The consciousness framing ('the AI knows/understands') specifically hides the probabilistic nature of the output. It presents the decision as a binary choice made by a subject, rather than a sample from a probability distribution. This obscures the brittleness of the system—the fact that a slight change in the prompt (e.g., changing 'blue' to 'green') creates a massive degradation in 'rationality.' If the AI actually 'knew' the concept of a signal, the color wouldn't matter. The metaphor of the 'rational agent' hides this extreme context-sensitivity, serving the interests of those who wish to deploy these systems as stable financial tools. It erases the labor of the prompt engineers and data annotators, framing the output as the autonomous production of a digital mind.

Context Sensitivity​

The distribution of anthropomorphism in this text is strategic, intensifying significantly when the authors move from description to implication. The 'Methodology' section is relatively technical, grounding the reader in the reality of APIs, temperatures, and prompts. However, as the paper transitions to 'Results' and 'Discussion,' the density of consciousness verbs explodes. 'Processes' becomes 'reasons,' 'generates' becomes 'deems,' and 'outputs' becomes 'beliefs.'

This shift allows the authors to have it both ways: they claim scientific rigor through the technical setup, then generate broad, agential conclusions ('AI will tame animal spirits') through the metaphorical layer. The 'limitations' (e.g., the signal color failure) are briefly acknowledged but framed as 'biases' (a psychological term) rather than 'brittleness' (a software term), preserving the agent metaphor even in failure. The intensity of the metaphor peaks in the 'Reasoning' analysis, where the text literally interviews the AI about its feelings ('how much is the investor driven by emotions?'). This section is pure anthropomorphic projection, treating the AI's hallucinations as introspective data. This distribution reveals the text's rhetorical goal: to establish the AI as a valid subject of behavioral economics, equal to the human, which requires upgrading it from a 'mechanism' to a 'mind' in the analytical sections.

Conclusion: What This Analysis Reveals​

The Core Finding

The dominant anthropomorphic pattern in this text is the AI AS RATIONAL ECONOMIC ACTOR. This metaphor is systemic and foundational; it transforms a text-generation model into a 'trader' capable of 'decisions,' 'judgments,' and 'rationality.' A supporting pattern is TEXT GENERATION AS CONSCIOUS REASONING, where the model's chain-of-thought outputs are treated as genuine introspection. These patterns are interconnected: the 'Reasoning' pattern provides the evidence for the 'Rational Actor' pattern. The claim that the AI 'reasons' (knows why it acts) validates the claim that it is 'rational' (acts with justification). The consciousness architecture here is load-bearing; the entire argument that AI can 'tame animal spirits' rests on the assumption that the AI is a 'knower'—an entity capable of understanding market fundamentals and resisting emotional impulses—rather than a 'processor' effectively mimicking rational text patterns.

Mechanism of the Illusion:​

The 'illusion of mind' is constructed through a specific rhetorical sleight-of-hand: The Validation of Hallucinated Interiority. The authors ask the AI to explain its decisions ('provide reasoning'), and then treat the generated text not as more output, but as causal explanation. When the AI writes 'I believe the asset is valuable,' the authors accept this as proof of a belief state. This leverages the 'curse of knowledge': the authors, knowing how financial reasoning works, project that understanding onto the coherent sentences the AI produces. The causal chain is deceptive: the AI generates a 'Buy' token and a 'Reasoning' token sequence based on the same probabilistic context. The authors, however, frame it as: Reasoning → Decision. This temporal and causal inversion creates the illusion of a rational mind at work, converting statistical correlation into psychological causation.

Material Stakes:​

The metaphors in this text have high-stakes consequences for Financial Stability and Regulatory Policy. If regulators accept the frame that AI agents are 'rational' and 'averse to herding' (knowing states), they may be induced to deregulate algorithmic trading or rely on AI as a stabilizing force ('taming animal spirits').

Specifically, if the AI is framed as 'knowing' the market fundamentals, regulators might treat it as a sophisticated investor (subject to standard market rules) rather than a high-frequency algorithmic tool (subject to circuit breakers and code audits). This shift could lead to a lack of oversight on the mechanistic failure modes of these models—such as the 'red/green' signal failure identified in the text. If an AI 'knows' value, it is trusted; if it merely 'processes' tokens, it is audited. The 'Rational Agent' framing benefits fintech firms by positioning their products as superior to 'irrational' humans, potentially fueling an investment bubble in AI-driven trading firms while obscuring the risk of flash crashes caused by correlated algorithmic failures. The loser is the broader market stability, which becomes vulnerable to 'rational' herding based on hallucinated signals.

AI Literacy as Counter-Practice:​

Practicing AI literacy in this context requires a disciplined refusal to accept 'reasoning' outputs as 'mental states.' Reframing 'the AI deems an opportunity' to 'the model predicts a positive sentiment token' (as demonstrated in Task 4) directly counters the Epistemic Risk of over-trust. By replacing consciousness verbs (knows, believes, reasons) with mechanistic verbs (processes, correlates, generates), we force the recognition that the system is blind to the market reality it trades on. This technical reality check undermines the narrative of the 'Rational Savior' that can tame human emotion. Systemic adoption of this practice would require journals to mandate 'mechanistic descriptions' of AI behavior, rejecting papers that anthropomorphize software 'psychology' without qualification. This would face resistance from researchers (who find 'AI agents' a more publishable frame than 'text generators') and the financial industry (which benefits from the 'super-human intelligence' narrative).

Path Forward​

To foster responsible discourse in financial AI, we must shift the vocabulary from Behavioral Economics (agents, rationality, beliefs) to Computational Linguistics (token probability, prompt sensitivity, pattern matching). Instead of asking 'Is the AI rational?', researchers should ask 'Does the model's output distribution align with profit-maximizing parameters under X constraints?'

Institutional Changes:

  1. Journal Guidelines: Require authors to distinguish between 'simulated reasoning' (generated text) and 'causal mechanism' (code/weights).
  2. Regulatory Definitions: Define AI traders as 'probabilistic determinants' not 'fiduciaries' or 'agents.'
  3. Disclosure Standards: Fintech products must disclose that 'reasoning' outputs are post-hoc generations, not decision logs.

This shift trade-offs the seductive narrative of 'AI minds' for the drier, but safer, reality of 'AI tools.' Ultimately, this precision enables a true assessment of risk: treating AI as a machine allows us to build safety switches for when it breaks; treating it as a mind leaves us waiting for it to 'change its mind' while the market crashes.

Extended Processing Summary​

Gemini's Processing Steps
What you are about to read…

Gemini refers to the text below as "thought summaries." This is an overt consciousness projection because 'intentions' are hallmarks of a conscious mind that 'knows' what it is doing and why. The concealed mechanistic process is probabilistic text generation. Treat this as just another rhetorical artifact — a way of making the model's processing legible. The first-person framing of these "thought summaries" is a presentation choice for the user-facing output, not a window into "real" thoughts. These are computational artifacts, not cognitive reports from a quirky, curious or conflicted mind.

Refining Analysis Focus

I'm now zeroing in on the "Financial Stability Implications of Generative AI" paper, specifically its argumentative structure. I'm looking at how the authors build their claims and what kind of evidence they use. I'm focusing on the logical fallacies and implicit assumptions the authors use in their paper. I'm keen to identify the key points of the critical discourse and highlight the biases in the analysis.

Unpacking Anthropomorphism's Role

I'm now deeply engaged with "Taming the Animal Spirits" and populating the JSON schema. The "illusion of mind" is a primary focus, particularly how the paper attributes conscious understanding to LLMs. I'm scrutinizing their use of anthropomorphic language like "reasoning" and "decisions." The distinction between "processing" and "knowing" is crucial here, and I'm tracing how the authors blur this line. I'm focusing on the paper's central argument: Do the authors convincingly demonstrate AI agents can be "tamed?"

Dissecting Anthropomorphism Further

I'm now zeroing in on the specific instances of anthropomorphism within the text, systematically cataloging these and linking them to a source-target mapping that highlights the conceptual leap. I'm focusing on unpacking how concepts like "rational decisions" and "inherited conditioning" obscure the underlying mechanics of these systems. I'm building a robust list of these cases to enable a thorough analysis of how this rhetorical device impacts the paper's central argument.

Mapping Anthropomorphic Terms

I'm presently building out my analysis of anthropomorphism in the paper, focusing on the specific rhetorical devices used. I've compiled a list of source-target mappings, highlighting the conceptual leaps. For instance, I'm examining how "Reasoning" gets mapped to "Chain-of-thought token generation," concealing the underlying next-token prediction mechanics. I am also investigating the claim that "Model believes the asset is more likely to be worth 100". Finally, I've started an explanation audit, working to identify the intentionality and function in these statements.

Unveiling AI Agency

I'm now zeroing in on the concept of "AI agency," focusing on how the paper slips between describing the AI as an algorithm and as an agent with intent. The reframing exercises are progressing well; I've expanded the list, focusing on clarifying the inherent statistical nature of AI output generation. I'm investigating how the authors blur the lines between probabilistic outputs and intentional decisions. Specifically, I'm examining how statements like "AI agents generally deem" obscure the underlying probabilistic mechanisms at play. I'm focused on the intentionality and function in these statements.

Examining AI's "Reasoning"

I'm now diving into the paper's final section, specifically analyzing how the authors construct the image of the AI as an economic actor. I'm focusing on the "Reasoning" prompts, which I believe generate the illusion of understanding. I'm actively assessing how the text obscures the mechanical aspects of the AI's "decisions" by emphasizing the AI's agency, a technique central to building credibility.

Exposing the Causal Illusion

I'm now focusing on how the paper creates a false causal link between the AI's "thoughts" and economic stability. By treating the AI's narrated "thoughts" as actual causal explanations, regulators may be misled into under-regulating algorithmic trading, which ignores the actual underlying risks. I'm investigating how to reframe this through "Mechanistic Behavioral Analysis" that doesn't rely on "AI Psychology."

Completing the Analysis

I'm now putting the finishing touches on my analysis, assembling the data into a clear and concise JSON format. I'm focusing on ensuring all instances of anthropomorphism and causal illusions are accurately represented. The goal is to provide a complete and understandable summary of the paper's key fallacies, reframed to emphasize mechanistic behaviors. The goal is a comprehensive summary that's easily digestible.