Robust Exploratory Stopping Under Ambiguity in Reinforcement Learning

Battle Count: 79

Highly relevant to quantitative trading. The paper directly addresses American option pricing (put and call) under model ambiguity, which is a core problem in derivatives trading. The robust framework handles model misspecification (e.g., dividend rate uncertainty), which is critical for real-world trading. The multi-dimensional basket and geometric option examples demonstrate scalability to practical trading scenarios. The RL algorithm provides a data-driven approach to optimal exercise decisions without requiring full model specification, making it applicable to algorithmic trading systems dealing with uncertain market dynamics.

High complexity. The implementation requires: (1) understanding of BSDE theory and g-expectation framework, (2) implementing the deep splitting method for PDEs with neural networks, (3) handling the quadratic variation-based estimation of volatility from observed data, (4) implementing the policy iteration loop with logistic-form policy updates, (5) managing the penalization parameter N and exploration temperature λ, (6) constructing the robust exploratory stopping time with the adjustment term. The theoretical underpinnings (reflected BSDEs, comparison principles, a priori estimates) are non-trivial.

Complete code is available on GitHub (https://github.com/GEOR-TS/ExploratoryRobustStoppingRL). All computations performed using PyTorch on Mac Mini with Apple M4 Pro. Hyperparameters (batch size, epochs, learning rate, network architecture) are specified. However, the theoretical proofs are extensive and the algorithm involves multiple interacting components (BSDEs, deep splitting, policy iteration) that require careful implementation.