Could Large Language Models work as Post-hoc Explainability Tools in Credit Risk Models?

Battle Count: 64

The paper is primarily focused on credit risk modeling and regulatory explainability rather than quantitative trading. However, the methodology of using LLMs as post-hoc explainability interfaces could be tangentially relevant to trading strategy explainability and risk model governance in financial institutions. The findings about LLMs being better as narrative interfaces than autonomous explainers have general implications for any ML model governance context in finance.

Moderate complexity. The pipeline involves: (1) standard data preprocessing and model training (logistic regression, XGBoost) using scikit-learn and XGBoost libraries; (2) SHAP computation for reference attributions; (3) prompt engineering with structured JSON output constraints; (4) LLM API calls via LiteLLM framework; (5) ranking comparison metrics (Overlap@K, Kendall's tau). The main complexity lies in careful prompt design, handling LLM output parsing, and the evaluation pipeline. No custom model training or novel algorithms are required.

The paper provides detailed prompt templates (Appendix A), specifies software versions (Python 3.12.12, pandas 2.2.2, NumPy 2.0.2, scikit-learn 1.6.1, SHAP 0.50.0, XGBoost 3.1.3, Optuna 4.6.0), uses a publicly available dataset (LendingClub from data.world), fixes temperature at 0 for LLM calls, and uses the LiteLLM framework (v1.81.3). However, no GitHub repository URL is explicitly provided, and the exact random seeds for data splitting are not stated. The evaluation uses 500 stratified test observations.