Reinforcement Learning in Queue-Reactive Models: Application to Optimal Execution

Battle Count: 72

Highly relevant to quantitative trading. The paper directly addresses the optimal execution problem, a core challenge for institutional traders. It provides a model-free RL framework that captures realistic market microstructure features (transient impact, liquidity resilience, order-flow imbalance) through the QRM simulator. The DDQN agent demonstrates significant outperformance over TWAP and rule-based benchmarks (up to 27% improvement). The methodology is directly applicable to algorithmic execution systems used by asset managers, hedge funds, and proprietary trading firms. The feature importance analysis and tactical adaptation findings provide actionable insights for execution algorithm design.

Moderate to high complexity. The DDQN architecture itself is relatively standard (5-layer feedforward network with 30 hidden units). However, the QRM simulator requires careful implementation of the 2K-dimensional Markov jump process with state-dependent intensities, invariant distribution sampling, and price dynamics with (θ, θ_reinit) parameters. Calibration of order-flow intensities from real data adds complexity. The training requires ~500,000 episodes with 25 decision points each. The action space design (percentage of best ask) and reward function with terminal penalty require careful tuning. No open-source code is provided, increasing implementation burden.

The paper provides detailed parameter settings (Table 1), algorithm pseudocode (Algorithm 1), QRM model specifications, and calibration parameters (θ=0.7, θ_reinit=0.85 from France Télécom data). However, no code repository is mentioned, and the QRM simulator implementation details (order-flow intensities) reference external work [29]. The training procedure is well-described but full reproducibility requires access to the calibrated QRM parameters and the specific simulation code.