Reinforcement Learning with Hawkes-Structured Networks for Market Making

By Fernando Acero, Philipp Plank, Yufei Zhang

Rating

1841
Battle Count: 51

Relevance

10/10
Highly relevant. The paper addresses a core problem in quantitative trading (market making) under realistic non-Markovian order flow dynamics (Hawkes processes). It proposes a novel, theoretically grounded, and empirically superior architecture for RL-based market making, directly applicable to high-frequency trading strategies.

Implementation Complexity

8/10
High complexity. Requires implementing a custom neural architecture (HSN) with learnable exponential filters, integrating it with a PPO reinforcement learning agent, and handling non-Markovian state representations. Theoretical understanding of Hawkes processes and Bernstein functions is beneficial for proper initialization and interpretation.

Reproducibility

3/5
The paper provides detailed mathematical formulations, architecture specifications, hyperparameters, and simulation settings (Section 5.1). However, no public code repository URL is provided in the text. The environment is fully specified as a simulation, allowing for potential reproduction if the code were available or implemented from scratch based on the detailed description.

Abstract

Market making under realistic order-flow dynamics is challenging due to the clustered, self-exciting nature of order arrivals, which are naturally modeled by non-Markovian Hawkes processes. We propose a model-free reinforcement learning (RL) framework for market making with general Hawkes order-flow dynamics. We first establish non-asymptotic performance bounds that quantify the loss incurred by approximating the infinite-dimensional Hawkes state using a finite collection of exponentially weighted features. Guided by this analysis, we introduce the Hawkes-Structured Network (HSN), a novel neural architecture that encodes historical order flow through a learnable mixture-of-exponentials filter. HSN is trained end-to-end using model-free RL, jointly optimizing the feature representation and quoting policy directly from reward feedback. Experiments on single- and multi-asset market-making problems with mixture-of-exponentials, power-law, and regime-switching kernels demonstrate that HSN outperforms general-purpose neural architectures, achieving higher trading rewards while implicitly recovering the underlying order-flow intensities.

Links

arXiv:2610.16524 ยท Paper rankings

The interactive Everscope explorer (charts, battles, favorites) loads below.