Calibration-Induced Degeneracy in LLM Financial Forecasting: An Audit-Trailed Case Study on Next-Day Market Risk

Battle Count: 72

The paper is highly relevant to quantitative trading methodology, particularly for teams considering LLM-generated features for risk forecasting. It provides a concrete, auditable case study demonstrating a common sequencing failure (acquiring expensive features before verifying calibration viability) and proposes a practical, low-cost checkpoint protocol. The finding that a near-zero-cost headline count outperformed expensive LLM semantic scoring on one endpoint is directly actionable for feature engineering decisions. However, the paper is primarily a methodological cautionary tale rather than a trading strategy, and the specific LLM score showed no validated predictive improvement.

The core methodology involves standard components: LLM API calls for headline scoring, logistic regression and QLIKE-based variance models with bounded parameters, moving-block bootstrap, and GARCH fitting. The proposed calibration-viability checkpoint is simple (fit, perturb, check threshold). However, the full experimental protocol with prespecification, hash-freezing, coverage gates, retry rules, familywise corrections, and audit trails adds significant procedural complexity. The $25.08 total API cost is modest, but the engineering discipline required for proper prespecification and verification is substantial.

The paper provides extensive audit trails, SHA-256 hashes, detailed procedural chronology, and same-author reimplementation verification (18, 17, 13, 13 checks passed for different analyses). However, the design was internally prespecified and hash-frozen rather than publicly preregistered. Verification was same-author, not external replication. Raw LLM outputs and headline text are not redistributed. The public archive supports recalculation from derived rows but not end-to-end pipeline replay. A new API run would constitute a new experiment. Exact dates leave LLM memorization unresolved.