Forecast collapse of transformer-based models under squared loss in financial time series

Battle Count: 70

Highly relevant to quantitative trading. The paper provides a rigorous theoretical explanation for why Transformer-based and other highly expressive ML models consistently fail to produce useful point forecasts of asset prices/returns under MSE loss. This directly informs strategy development: practitioners should not expect architectural improvements to yield predictive edge in financial time series under standard loss functions. The findings suggest redirecting ML efforts toward distributional forecasting, alternative objectives, or decision-theoretic frameworks rather than point prediction. The EUR/USD empirical results (PatchTST error ratio ~1.71x linear, 92% of windows worse) provide concrete evidence for practitioners evaluating ML-based trading signals.

The theoretical framework is mathematically rigorous but conceptually straightforward (conditional mean under squared loss, bias-variance decomposition). The empirical component uses a standard PatchTST configuration with well-defined hyperparameters (L=451, H=30, 30-second data). Reproducing the numerical experiments requires access to high-frequency FX data and a standard Transformer training pipeline. The main complexity lies in the theoretical proofs (martingale concentration, Glivenko-Cantelli arguments) rather than implementation.

The paper provides complete theoretical proofs (Propositions 1-2, Corollary 1, Appendix A.1-A.2) and describes the experimental setup in detail (PatchTST configuration, data splitting, L=451, H=30, 30-second EUR/USD data). However, no code repository is mentioned, and the specific PatchTST hyperparameters beyond the standard configuration are not fully enumerated. The dataset is described but not explicitly linked to a public source.