Behavioral Consistency Validation for LLM Agents: An Analysis of Trading-Style Switching through Stock-Market Simulation

Battle Count: 50

The paper is highly relevant to quantitative trading research as it validates whether LLM-based agents can replicate real-world trading behavior (style switching between fundamental and technical approaches). However, it is more of a validation/framework paper than a direct trading strategy paper. It informs the design of more realistic agent-based simulations for backtesting and understanding market dynamics. The findings that LLM agents are only partially consistent with behavioral finance theories have direct implications for using LLM agents in trading simulations.

The framework requires: (1) setting up a multi-agent simulation with 32 agents across 253 trading days, (2) implementing daily trading decisions with technical/fundamental indicators, (3) maintaining dual ledgers (actual and counterfactual), (4) periodic style-switching evaluations every 10 days, (5) computing four alignment scores, (6) performing Mann-Whitney U tests with effect sizes, and (7) managing LLM API calls for 32 agents × 253 days × multiple models. The prompt engineering for behavioral traits and the factorial design add complexity. No open-source code is provided.

The paper provides detailed prompts (Appendix B), simulation setup parameters, data fields (Table 1), and indicator formulas (Table 2). However, no code repository is mentioned. The use of proprietary LLM APIs (GPT-4o-mini, Gemini) introduces non-determinism. Stock data (S&P 500 2024) is publicly available. The factorial design and evaluation framework are well-described but full implementation details are partially deferred to appendices.