The Inference-Compute Frontier and a Latency-Efficient Architecture for Limit Order Book Prediction

Battle Count: 51

Highly relevant to quantitative trading, particularly for high-frequency and market-making strategies. The paper directly addresses the critical tradeoff between model accuracy and serving latency in LOB prediction, which is central to whether a mid-price movement signal is actionable. The inference-compute frontier provides a principled framework for capacity allocation in trading ML systems. FastBiNLOB demonstrates that SOTA accuracy can be achieved at significantly lower latency (23-60% reduction), which is directly valuable for latency-sensitive trading. The separation of structural work from latency as design targets is practically important for HFT infrastructure decisions.

FastBiNLOB is architecturally simple: BiN normalization, linear embedding, two residual axis-separable blocks (feature MLP + temporal MLP), temporal reducer, and per-horizon heads. No attention, no complex graph structure. The frontier analysis requires careful structural-work accounting across multiple model families. The deployment pipeline includes multi-task training, logit-adjusted cross-entropy, per-horizon fine-tuning, and class-bias calibration. The main complexity lies in the experimental infrastructure (2520+ model configurations, cross-fold evaluation, latency timing harness) rather than the model itself.

Code is publicly available on GitHub (https://github.com/evanshedges2/LOB_scaling_and_FastBiNLOB). FI-2010 is a standard public benchmark. The paper provides detailed structural-work counting conventions, explicit hyperparameters for FastBiNLOB variants, five-seed experiments with standard errors, and a specific hardware/software environment (MacBook Pro M3 Max, PyTorch 2.12.0, eager FP32). However, the scaling lane uses raw LOB40 features while the deployment lane uses full 144 features, and the two lanes are not directly comparable. The latency measurements are hardware-specific (Apple M3 Max CPU/MPS).