From Accuracy to Auditability: A Survey of Determinism in Financial AI Systems

Battle Count: 50

The paper focuses on regulated financial AI systems (credit scoring, fraud detection, AML) rather than quantitative trading. While it discusses backtest overfitting and factor zoo as historical context, the core contribution is about computational determinism and auditability in compliance-driven ML deployments. The findings on model reproducibility could indirectly affect trading model validation, but the paper does not address trading strategies, portfolio optimization, or market prediction directly.

The paper addresses systems-level challenges spanning hardware (GPU atomic operations, tensor parallelism), frameworks (PyTorch Geometric, vLLM, FlashAttention), model architectures (GNNs, LLMs, tree ensembles), and regulatory compliance infrastructure. Implementing the proposed solutions (batch-invariant kernels, neuro-symbolic gateways, cryptographic audit trails, deterministic scatter operations) requires deep expertise across multiple systems layers. The memory-determinism trade-off (5× VRAM for deterministic GNN aggregation) and the need for pre-provisioned logprob retention add operational complexity.

The paper itself is about reproducibility and determinism. It uses public datasets (UCI German Credit, Kaggle Credit Card Default, Elliptic Bitcoin) and open-source models (Qwen2.5-7B, InternLM2.5-7B, Phi-3-mini). Experiments are described with specific configurations (50 seeds × 10 splits, 30 runs per instance, 10 runs per config). However, no code repository is explicitly linked. The v3 version corrects multiple errors from v1-v2, demonstrating iterative reproducibility challenges. The paper acknowledges that faithful TDI measurement requires logprobs not retained in recorded runs.