Beyond Agent Architecture: Execution Assumptions and Reproducibility in LLM-Based Trading Systems

Battle Count: 164

Directly addresses the core evaluation infrastructure of quantitative trading research. The paper audits 30 LLM-based trading studies for execution realism, transaction costs, timing assumptions, and reproducibility. The worked example demonstrates how cost sensitivity (0, 10, 25 bps) materially changes strategy rankings. The reporting checklist is directly actionable for quantitative trading researchers. The paper identifies that architecture reporting outpaces evaluation discipline, which is a critical bottleneck for translating LLM trading research into practice.

The review methodology itself is straightforward (coding schema, evidence matrix, screening protocol). The worked example uses a simple deterministic score formula with moving averages, momentum, event proxies, and VIX macro signals. No actual LLM API calls are made in the scaffold. The main complexity lies in the systematic coding of 30 studies across 5 dimensions, which is labor-intensive but technically simple. The reporting checklist is immediately actionable.

The paper provides a transparent screening log (PRISMA-style), a coded evidence matrix with 30 studies, a coding rulebook, author validation with 150 field-level checks, and a reporting checklist. The worked example uses public data (FRED VIX, local OHLCV CSVs) and deterministic scaffold. However, many reviewed studies lack artifact release (only 18/30 recoverable), and the worked example is intentionally limited to 10 stocks. The paper itself is highly reproducible as a review methodology.