Athenarabeta

athenara:~$ registry bench benchmarks/finben

FinBen

A holistic open-source financial benchmark of 42 datasets spanning 24 tasks, including the first stock-trading evaluation for LLMs.

RepositoryWebsiteMarkdownEdit this entry

#benchmark #financial-llm #evaluation #stock-trading #rag

added 2026-08-15 · external

$ git clone https://github.com/The-FinAI/FinBen
collected 8 task environments
finben::information-extraction
finben::textual-analysis
finben::question-answering
finben::text-generation
finben::risk-management
finben::forecasting
finben::decision-making
finben::stock-trading

FinBen evaluates LLMs across eight aspects: information extraction, textual analysis, question answering, text generation, risk management, forecasting, decision-making, and bilingual English/Spanish coverage. It introduced the first stock-trading evaluation for financial LLMs plus novel agent and retrieval-augmented generation evaluations, and released new datasets for regulations and stock trading.

The authors evaluated 21 representative LLMs (including GPT-4, ChatGPT, and Gemini), finding strong performance on extraction and textual analysis but weakness on advanced reasoning, generation, and forecasting. Published in the NeurIPS 2024 Datasets and Benchmarks track (arXiv:2402.12659); it grew out of the FLARE benchmark (see related). The repository carries no license file at time of indexing.

trading [●●···] basic   ai [●●●●·] advanced   programming [●●●··] moderate   setup [●●●··] moderate

authors Qianqian Xie, Weiguang Han, Zhengyu Chen, Ruoyu Xiang, Jimin Huang, The FinAI, et al.
origin external
markets equities

builds on pixiu-flare

extended by StockBench: Can LLM Agents Trade Stocks Profitably In Real-world Markets?

athenara:~$