VAIOM: Continuous-Input, Discrete-Output Decoder-Only Financial Sequence Modeling

Battle Count: 58

The paper directly addresses probabilistic next-return modeling for FX and spot metals, which is core to quantitative trading. The focus on full conditional distributions (not point forecasts) is highly relevant for risk management, position sizing, and tail-risk estimation. The tail-bucket decomposition showing concentrated gains in extreme events is particularly valuable for risk-aware strategies. However, the paper explicitly notes that likelihood compression is not trading profitability, and no portfolio construction, transaction costs, or execution modeling is included. The 1H frequency and FX/spot-metal scope limit direct applicability to other asset classes. The model is compact (0.9M parameters) which is practical for deployment.

The architecture is relatively compact (0.9M parameters, 4-layer Transformer) but involves multiple components: continuous input projection MLPs, categorical metadata embeddings, fusion MLP, MoMS output head with 4 latent components, three auxiliary heads (Gap, VolReg, Ordinal), full-sequence supervision with per-task masking, and complex preprocessing (EWMA volatility scales, bucket construction, data-quality masks). The training pipeline requires careful chronological split management, causal information flow, and multi-seed evaluation. FP16 mixed precision and gradient accumulation add engineering complexity. The bucket construction and tokenizer serialization require careful implementation to avoid leakage.

The paper provides detailed configuration (Table 11), exact architecture specifications, optimizer settings (AdamW, LR 3e-4, weight decay 0.01), training seeds (17, 29, 43), FP16 mixed precision, gradient accumulation details, and complete bucket edge values. Multi-seed robustness is tested. However, no GitHub repository or code link is explicitly provided. The corpus is derived from Dukascopy one-minute FX data. All preprocessing parameters are fitted only on Train and serialized.