StockBench: Can LLM Agents Trade Stocks Profitably In Real-world Markets?

Each model starts with $100,000 in cash and no holdings and makes daily decisions at market open across the 20 highest-weighted Dow Jones constituents, from 3 March to 30 June 2025 — 82 trading days. The daily prompt carries the agent’s actions on held stocks over the past seven days, up to five news articles from the previous 48 hours, and fundamentals such as market capitalisation, P/E ratio, dividend yield and the 52-week range. Every agent runs three times under different seeds and the average is reported. Scoring is on final return, maximum drawdown and Sortino ratio against a passive equal-weight buy-and-hold portfolio held unchanged across the same window.

Read the paper’s table rather than its abstract. The abstract states that most models struggle to outperform buy-and-hold, while the body and results table of the same version report the opposite: most of the thirteen tested models finish above the baseline’s 0.4% return, and all of them post a smaller maximum drawdown than it. An ablation on the best-scoring model shows its return falling as news, and then fundamentals, are removed from the prompt.

The repository ships the evaluation data rather than fetching it — per-date parquet files with SHA-256 checksums for the 20 targets plus SPY, alongside cached news, financials and indicator JSON — so the paper’s window can be rerun without a data vendor account, while extending it to other periods or symbols needs Polygon and Finnhub keys plus an LLM key. The code has been dormant since October 2025: the benchmark package has a single commit and every later one touches only the README. The Apache-2.0 LICENSE is unmodified boilerplate with the copyright holder never filled in, and the repository names no owning institution — the Tsinghua and BUPT affiliation comes from the paper. Results rest on one 82-trading-day window in a mildly positive market, with no significance testing across the three seeds, and the “contamination-free” property is a design argument about post-cutoff data rather than a measurement, one that decays as newer models are evaluated.