PriceSeer: Evaluating Large Language Models in Real-Time Stock Prediction

Battle Count: 62

Directly relevant to quantitative trading as it evaluates LLMs' ability to predict stock prices and generate investment strategies. However, it is primarily a benchmark/evaluation paper rather than proposing a new trading model. The findings about sector-specific predictability, prediction horizon effects, and fake news vulnerability are practically useful for quant traders considering LLM-based tools. The hit rates (~0.5-0.7) and relative errors (2-5%) suggest LLMs are not yet reliable enough for direct trading deployment but show potential as supplementary tools.

The benchmark itself is relatively straightforward to implement: collecting historical stock data via yfinance, computing standard financial indicators (SMA, RSI, MACD, Bollinger Bands), gathering news via SerpAPI, and prompting LLMs with structured templates. The main complexity lies in the real-time data collection pipeline and the multi-modal prompt construction. No model training is required since it evaluates existing LLMs via API calls.

Code and evaluation data are stated to be open-sourced at GitHub. The benchmark uses publicly available data from Yahoo Finance and Google News via SerpAPI. Six commercial LLMs are evaluated via their APIs, which may change over time. The methodology is well-documented with clear formulas for financial indicators and evaluation metrics.