Robustness in Sequential Decision Making Under Evolving Uncertainty: Evidence from High-Frequency Market Making

By Ying Chen, Hoa Nguyen, Julian Sester, Hoang Hai Tran, Yijiong Zhang

Rating

1624
Battle Count: 78

Relevance

9/10
Highly relevant to quantitative trading. The paper directly addresses high-frequency market making, a core quantitative trading strategy. It provides a principled framework for handling model uncertainty in sequential trading decisions, with empirical evidence on when robustness creates value (high-liquidity markets) and when it can be harmful (illiquid markets). The decomposition into uncertainty tolerance and action robustness provides practical levers for practitioners. The simulation and empirical results on real market data (AAPL, TSLA, MKC, TWLO) across stable and stressed periods (2019 vs 2020 COVID) are directly applicable to live trading systems.

Implementation Complexity

8/10
High implementation complexity. The framework requires: (1) training a deep ensemble model for transition dynamics, (2) implementing Sinkhorn optimal transport with entropic regularization, (3) solving the robust Bellman equation via a three-network actor-critic architecture (critic, actor, lambda networks), (4) constructing rolling-window state representations with 13+ features, (5) calibrating simulation environments from high-frequency data, and (6) managing the dual optimization over lambda. The mathematical machinery (Sinkhorn duality, dynamic programming under distributional ambiguity) requires significant expertise. However, the paper provides detailed algorithms and hyperparameters that aid implementation.

Reproducibility

3/5
The paper provides detailed algorithm descriptions (Algorithms 1 and 2), hyperparameter tables (Table 5), simulation parameter calibration (Table 4), and mathematical formulations. However, no code repository is mentioned. The simulation environment is calibrated from real AAPL data, and empirical analysis uses publicly available high-frequency data for AAPL, TSLA, MKC, and TWLO. The deep ensemble model architecture and training details are described but specific implementation code is not provided.

About this paper

Methodology: Robust Reinforcement Learning with Sinkhorn Ambiguity Sets. Problem types: Market Making, Reinforcement Learning, Sequential Decision Making, Risk Management, Optimization, Distributionally Robust Optimization.

The interactive Everscope explorer (charts, battles, favorites) loads below.