Same Text, Different Numbers: The Divergence of LLM-Based Measures

Battle Count: 0

Highly relevant for researchers and quants using LLMs to generate alpha signals from text. It highlights that the choice of LLM provider is a critical hyperparameter that can flip the sign or significance of trading signals derived from sentiment or risk measures.

The methodology is straightforward (scoring text with APIs and comparing outputs), but requires access to multiple LLM APIs and significant computational resources to score large datasets consistently.

The paper provides detailed prompts, model identifiers, and data sources (S&P Capital IQ, Compustat, IBES, CRSP). However, the specific earnings call transcripts are proprietary and not publicly available, limiting full external replication without access to the same data.