Detecting AI Hallucinations in Finance: An Information-Theoretic Method Cuts Hallucination Rate by 92%

Battle Count: 89

The paper is tangentially related to quantitative trading through its focus on financial QA and hallucination detection in LLM-based decision support. However, it does not address trading strategies, market prediction, portfolio optimization, or risk modeling directly. The primary contribution is a general-purpose hallucination detection mechanism applicable to any LLM deployment in finance, which could indirectly support trading research by ensuring reliability of LLM-generated financial analysis. The 92% hallucination rate reduction at 30% coverage could be relevant for LLM-assisted trading systems where incorrect financial facts could lead to erroneous decisions.

Moderate complexity. The core detector is a logistic regression on 7 features, which is straightforward. However, the feature extraction pipeline requires: (1) multi-sample generation with fact clustering using spaCy NER and regex, (2) API calls with logprobs=True for perplexity scoring, (3) contradiction detection via fact extraction, and (4) careful handling of numeric tolerances. The theoretical framework (Theorem 4) is not directly implemented but serves as design prior. Total API cost is ~$0.01 per example (12 calls). The semantic clustering procedure is domain-specific and would need adaptation for other domains.

Authors plan to release the financial QA dataset, ECLIPSE implementation, and all experimental code upon acceptance. Hyperparameters are fully specified (Table 9). The method uses standard APIs (OpenAI logprobs) and scikit-learn logistic regression. However, the dataset is small (n=200), synthetic, and domain-specific. The semantic clustering procedure is heuristic (spaCy NER + regex). No GitHub repository is currently available.