Right Place, Right Time: Market Simulation-based RL for Execution Optimisation

Battle Count: 67

This paper is highly relevant to quantitative trading as it directly addresses optimal trade execution, a core problem in algorithmic trading. It provides a practical framework for discovering execution strategies that minimize slippage while balancing risk and impact. The integration with the Almgren-Chriss efficient frontier provides a theoretically grounded evaluation metric. The decomposition of slippage into impact and risk components is directly applicable to execution algorithm design. The interpretability focus (learning parameters of known algorithm templates) aligns with regulatory requirements (EU AI Act). The results demonstrate consistent outperformance over TWAP/VWAP baselines.

The RL component itself is relatively simple (feed-forward network, REINFORCE, multi-armed bandit formulation). However, the primary complexity lies in the agent-based market simulator (Simudyne Pulse), which requires: (1) exchange matching engine implementation, (2) multiple trader agent types (Chiarella model with fundamental, momentum, noise, market maker agents), (3) calibration to historical data using surrogate model approaches, (4) aggregate impact function fitting, and (5) baseline-counterfactual simulation for risk/impact decomposition. The simulator is proprietary, making full reproduction difficult. The RL training converges quickly (within 20 episodes), but the overall pipeline requires significant infrastructure.

The methodology is described in detail, including network architecture, hyperparameters (lr=0.01, batch size=10, 50 MC samples, entropy coefficient β=2.0), and calibration procedures. However, the Simudyne Pulse market simulator is proprietary and not publicly available. The HKEX Hang Seng Index Futures data (Sept 11-12, 2023) is referenced but not explicitly provided. No code repository is mentioned. The Chiarella model parameters are provided in Table 1, but full simulator implementation details are deferred to other publications [17, 19].