The Virtue of Sparsity in Complexity

Battle Count: 88

Highly relevant to quantitative trading. The paper directly addresses how to construct optimal portfolios via SDF estimation in high-dimensional settings. The finding that sparse selection (basis pursuit) outperforms dense ridgeless estimation at high complexity has direct implications for factor-based portfolio construction, signal extraction, and alpha generation. The near-pathwise dominance and welfare gains are practically meaningful for portfolio managers. The distinction between capacity sparsity and factor sparsity provides a framework for designing factor models in practice.

The methodology is conceptually clear but computationally demanding. Key steps include: (1) generating RFF features (P up to 30,000) from 130 base characteristics, (2) solving basis pursuit (ℓ1 minimization) in overparameterized settings (P>>T), (3) rolling-window estimation over 360 months, (4) 20 independent RFF draws per complexity level. The ℓ1 optimization in high dimensions requires specialized solvers (e.g., LARS, coordinate descent, or interior-point methods). The grid search over P and σ adds computational cost. However, the core algorithms (RFF, basis pursuit, ridgeless regression) are well-established and available in standard ML libraries.

The paper uses publicly available JKP data (jkpfactors.com), specifies all hyperparameters (T=60, P grid, σ grid, 20 RFF draws), and follows the empirical design of Didisheim et al. (2025). However, all quantitative results are presented graphically (Figures 2-9) rather than in tabular form, making precise numerical reproduction slightly harder. The RFF random seed procedure is described but not fully deterministic. No code repository is mentioned.