Multi-Level Market Making with Reinforcement Learning

Battle Count: 68

This paper is directly and highly relevant to quantitative trading. It addresses the core market-making problem in limit order books, which is fundamental to electronic trading. The RL framework handles realistic multi-level order placement, inventory management, and adverse selection - all critical challenges for quantitative market makers. The paper provides concrete numerical results showing significant outperformance over heuristic benchmarks.

The implementation requires: (1) a multi-agent Poisson order-flow market simulator, (2) a deep-set encoder for variable-length order features, (3) logistic-normal distribution sampling and gradient computation, (4) actor-critic training with combined loss, (5) Hamilton apportionment for integer rounding, (6) potential-based reward shaping, and (7) feature normalization. The mathematical machinery (logistic-normal density, policy gradients) is non-trivial. However, the neural network architectures are relatively simple (2 hidden layers, 128 nodes). The main complexity lies in the simulation environment and the distribution-specific gradient computations.

The paper provides detailed hyperparameters (Table 4), simulation parameters (Table 5), network architectures, initialization schemes, and feature normalization procedures. However, no code repository is mentioned. The simulation environment is based on Poisson order-flow modeling from prior work (Cont et al., 2010; Cheridito and Weiss, 2026). All benchmark algorithms are fully specified. The main barrier to reproduction is the absence of source code.