LiveTradeBench: Seeking Real-World Alpha with Large Language Models

Battle Count: 56

Highly relevant to quantitative trading as it directly evaluates LLM-based agents in live trading environments with portfolio management abstraction. It provides empirical evidence on trading performance metrics (Sharpe ratio, drawdown, returns), demonstrates that general LLM capability does not translate to trading skill, reveals distinct portfolio management styles across models, and introduces a framework for evaluating sequential decision-making under real market uncertainty. The multi-market evaluation (stocks + prediction markets) and live data streaming approach are particularly relevant for practitioners seeking to understand LLM capabilities in trading contexts.

Moderate complexity. The framework provides an open-source Python package with simple APIs for data collection, environment setup, and agent construction. The agent architecture uses a standard ReAct framework with tool use and memory. However, setting up live data pipelines (stock prices, Polymarket CLOB, Google News), managing the POMDP environment, handling multi-market evaluation, and ensuring proper prompt engineering for 21 different LLMs adds operational complexity. The core logic is straightforward but the live infrastructure requires ongoing maintenance.

The paper provides an open-source Python package (live-trade-bench on PyPI), a GitHub repository (github.com/ulab-uiuc/live-trade-bench), a live web application (trade-bench.live), detailed prompt templates in appendices, and comprehensive model configuration details. However, the live nature of the evaluation means exact reproduction of results is inherently limited by market dynamics. The 50-day evaluation period (Aug 18 - Oct 24, 2025) is fixed but not repeatable.