Counterfactual Analysis via Large Language Models

Battle Count: 69

The paper is primarily about counterfactual analysis in online lending rather than trading. However, the methodology of using LLMs for counterfactual reasoning and the human-machine collaboration framework could be relevant for evaluating alternative trading strategies, risk scenarios, and decision-making under uncertainty. The ROI prediction and interest rate analysis have tangential connections to fixed-income and credit markets.

The core methodology involves API calls to GPT-3.5 with carefully engineered prompts, which is relatively straightforward. However, the tree-of-thought prompting with multiple experts, the forecast encompassing test, ROI calculation from loan status/duration predictions, and the integration with ML models (XGBoost, gradient-boosted regression, mixture cure model) add moderate complexity. The main challenge is prompt design and managing API costs at scale.

The paper uses GPT-3.5 via OpenAI API and LendingClub public data. However, the exact prompts are shown in figures, and the alternative interest rates come from Gopal et al. (2024) working paper. The 10,000-sample subset for prediction and full test set for counterfactuals are described. No code repository is mentioned. The stochastic nature of LLM outputs adds variability.