Uncovering Representation Bias for Investment Decisions in Open-Source Large Language Models

Battle Count: 67

Highly relevant for quantitative trading workflows that incorporate LLM-generated signals or views into investment decisions. The paper demonstrates that LLM confidence systematically favors large-cap, high-visibility firms and certain sectors (Technology, Communication Services), which could introduce unintended biases into factor models, stock selection, and portfolio construction. The finding that LLM preferences align best with fundamentals (free cash flow) and moderately with technical signals (volume) but poorly with growth indicators provides actionable guidance for integrating LLM outputs into mean-variance optimization or Black-Litterman frameworks. The sector-specific anchoring effects and cross-context instability (especially in Technology) are critical for risk management in LLM-assisted trading systems.

Moderate complexity. The core methodology involves pairwise prompting of LLMs with constrained decoding, which is straightforward to implement. However, the full pipeline requires: (1) assembling and standardizing ~150 firms' financial features across multiple categories, (2) implementing balanced round-robin comparison with order and repetition controls, (3) computing token-level log-probability confidence scores, (4) performing multi-method statistical analysis (Pearson, Spearman, Kendall correlations with FDR correction, ANOVA, logit-scale dispersion). The statistical analysis is standard but the data preparation and feature engineering for financial data adds complexity. No code is provided, requiring reimplementation.

The paper provides detailed prompt templates (Table 5), feature lists (Tables 2-3), sector/industry classifications (Table 4), and statistical methodology. However, no GitHub repository or code is explicitly provided. The dataset of ~150 U.S. firms (2017-2024) with standardized financial features is described but not linked to a specific public source. The Qwen models are publicly available on HuggingFace. Replication would require reconstructing the exact firm universe and feature standardization pipeline.