TT-DAC-PS: Twin-Target Deterministic Actor-Critic with Policy Smoothing for Optimal Trade Execution

Battle Count: 110

Directly addresses optimal trade execution, a core problem in quantitative trading and institutional finance. The paper provides a novel RL architecture specifically designed for episodic liquidation under market impact constraints, benchmarked against industry-standard strategies (TWAP, VWAP, AC). The methodology is highly applicable to algorithmic execution desks, with practical considerations like participation caps, LOB preprocessing, and safety constraints. The use of real NASDAQ LOB data for 10 US stocks adds practical credibility.

The architecture combines multiple components: twin EMA target critics with pessimistic backup, TD3-style policy smoothing, delayed actor updates, conservative Q regularisation, hybrid adaptive OU exploration with three modulation mechanisms (deterministic decay, variance-aware adjustment, learned temperature), AC utility-based reward shaping, and robust LOB preprocessing (MAD filtering, IQR capping). While each component is individually well-understood, integrating them into a coherent training loop with proper constraint handling requires careful engineering. The custom PyTorch implementation with Gym wrapper adds complexity. Network architecture is relatively simple (2 hidden layers).

The paper provides a detailed reproducibility checklist (Table 1) including random seed (0), model architecture (2 hidden layers: 24 and 48 units, ReLU, tanh output), optimizer (AdamZ with specific learning rates), training setup (replay buffer 10^4, batch 128, gamma 0.99), risk aversion parameter (lambda=10^-5), hardware (NVIDIA H100 GPU), and software stack (Python, PyTorch, Stable-Baselines3, Gymnasium). Full algorithm pseudocode is provided. However, no code repository is linked and the specific LOB data preprocessing pipeline details could be more explicit.