Portfolio Optimization Under Recursive Utility via Reinforcement Learning

Battle Count: 127

Directly addresses portfolio allocation using RL with a theoretically grounded risk-sensitive objective from asset pricing. The recursive utility framework separates risk aversion from intertemporal substitution, which is economically meaningful for portfolio management. Results show improvement over naive discounted RL baselines in Sharpe ratio and drawdown. However, the method underperforms simple equal-weight benchmark, and the experimental setup (single market, no transaction costs, one trial per split) limits practical applicability. The theoretical contribution (CE-based value target, AAE) is novel and relevant for risk-aware RL in finance.

Requires understanding of recursive utility theory (Epstein-Zin), certainty equivalent computation via Monte Carlo, non-linear Bellman operators, and approximate advantage estimation with state-dependent weights. The actor-critic framework (PPO/A2C) is standard, but the recursive utility value target and AAE add significant complexity. The simplex-constrained action space with incremental parameterization and projection adds engineering overhead. Hyperparameter tuning for gamma, psi, beta, kappa, and K (CE samples) is non-trivial. The ablation study shows high sensitivity to training window and K, requiring careful calibration.

The paper provides detailed derivations in appendices, specifies hyperparameters (beta=0.99, gamma=5.0, psi=1.0, kappa_init=0.1, ce_samples=10, PPO clip_eps=0.2, lambda=0.95), describes the environment and data preprocessing, and uses publicly available South Korean ETF data via pykrx and KIS developer API. However, no GitHub repository or code link is provided. Only one trial per split is reported, and high variance across splits is acknowledged. The data file path (data/data.csv) is mentioned but not publicly linked.