DATEDGPT: Preventing Lookahead Bias in Large Language Models with Time-Aware Pretraining

Battle Count: 160

Directly addresses a critical methodological issue in quantitative trading: lookahead bias in LLM-based backtesting. The paper quantifies the lookahead premium (26.4 b.p. per SD, significant at 1% level) and demonstrates that lookahead-bias-free models still achieve meaningful predictive power (Sharpe ratio 3.20, 28.3 b.p. economic magnitude). The framework enables honest evaluation of LLM-based trading signals and provides a practical tool for detecting inflated backtest results. The DATEDGPT series can serve as an auxiliary signal for financial decision systems.

Training 12 models from scratch requires approximately 24,000 GPU hours total (2,000 per model on A100s). The architecture follows standard Llama/GPT-2 reproduction (Karpathy's llm.c). Data curation involves multi-step pipeline: FineWeb-Edu filtering by crawl date, Qwen3-4B quality scoring, fastText classification, DeepSeek-V4-Pro QA generation with lexical constraints, and rule-based verification. Instruction tuning adds 3 epochs per model with 10% pretraining data mixing. The complexity is moderate for the ML pipeline but the compute requirement for 12 models is substantial.

Full model series (twelve models) released on HuggingFace (hf.co/datedGPT), DATEDINSTRUCT dataset released, interactive web demo available at www.datedgpt.com, detailed hyperparameters provided in appendix, training curves shown, and prompts for data curation documented. Each pretraining round requires approximately 2,000 GPU hours on A100 GPUs.