MLP-Initialized Reinforcement Learning for Exit Timing in SPX Short-Straddle Portfolios

By Jianning Wang, Robert Ślepaczuk, Paweł Sakowski

Rating

1846

Battle Count: 50

Relevance

9/10

Highly relevant for practitioners dealing with short-volatility strategies. It provides a robust framework for exit timing, demonstrating that supervised initialization is critical for RL success in financial markets with sparse rewards and high noise.

Implementation Complexity

7/10

Requires implementing a custom trading environment with realistic transaction costs, bid-ask spreads, and portfolio accounting. The two-stage training pipeline (Supervised -> RL) adds complexity compared to single-stage models.

Reproducibility

4/5

Code is available on GitHub. However, the underlying SPX option quotation data is proprietary (licensed from Cboe) and not included in the public repository, limiting full reproduction without institutional access to equivalent data.

Abstract

This paper studies exit timing for SPX short-straddle portfolios using a two-stage supervised and reinforcement-learning framework. Historical episodes first generate ex-post optimal hold/exit labels for training a multilayer perceptron (MLP) using only contemporaneously available state variables at deployment. The trained MLP then initializes reinforcement-learning actors that are fine-tuned with a risk-aware trading reward. Out-of-sample evaluation over 2024-2026 shows that the supervised MLP substantially outperforms passive holding and representative fixed-time and profit-and-loss-based exit rules. Across seven independent seeds, MLP-initialized Actor-Critic averages 37.54% cumulative return and a Sharpe ratio of 1.705, compared with 35.01% and 1.651 for the supervised MLP. Actor-Critic trained from scratch averages only 0.18% return. These findings show that oracle-guided supervised learning provides a stable foundation for reward-based policy fine-tuning in a sparse and path-dependent option-exit problem.

Links

arXiv:2610.13126 · Code · Paper rankings

The interactive Everscope explorer (charts, battles, favorites) loads below.