Trade-R1: Bridging Verifiable Rewards to Stochastic Environments via Process-Level Reasoning Verification

Battle Count: 85

Directly addresses stock selection as a quantitative trading problem using RL with LLM-based reasoning. The framework explicitly targets portfolio construction, excess return optimization relative to benchmarks (CSI-300, S&P 500), and cross-market generalization. The reward hacking problem in financial RL is a core challenge in quantitative trading. The paper provides concrete portfolio construction methodology (market-cap weighting, tranche-based rolling strategy, 10-day holding period) and evaluates on real market data with standard financial metrics (Sharpe, MDD, cumulative return).

High complexity due to multiple interacting components: (1) RL training loop with GRPO requiring group sampling and advantage normalization, (2) Two-stage RAG verification pipeline with embedding-based retrieval and LLM judge scoring, (3) Triangular consistency metric requiring three separate pairwise evaluations, (4) Asymmetric reward gating with conditional logic, (5) Belief augmentation generating 15 variants per day, (6) Portfolio construction with market-cap weighting and tranche-based rolling. Requires access to LLM judge APIs, embedding models, and substantial GPU resources for full-parameter training.

The paper provides detailed implementation specifics including model architecture (Qwen3-8B-Instruct), training hyperparameters (batch size 32, LR 1e-6, G=8 rollouts), embedding model (BGE-M3), judge model (Doubao-seed-1.8), and dataset construction pipeline. However, no code repository is linked, and the proprietary judge model (Doubao-seed-1.8) may limit exact reproduction. The 15 investment belief prompts are fully listed in the appendix.