Quant-Arena benchmark: no LLM beats the equal-weight market on A-shares in 2024
Quantitative research is an iterative loop: a researcher proposes a market hypothesis, turns it into an executable signal, tests it on historical data, and decides what to keep. The authors of a new preprint argue that existing benchmarks score formulas, code, or individual trading decisions, which leaves open whether large language models can complete the whole loop and produce strategies that still work beyond the research period.
Their answer is Quant-Arena, a benchmark and public leaderboard for end-to-end quantitative research on real A-share market data. A model builds features and stock-scoring code. It receives feedback only on 2022 and 2023, then freezes a single strategy, which is assessed on the sealed year 2024.
Assessment runs through a cash-account evaluator. It applies trading constraints and costs, including market impact estimated from a simulated order book. Scores are dominated by realized excess return, and they are complemented by signal quality and by the diversity of features the strategy uses.
The authors evaluate nine frontier models. GPT-6 Astra scores highest, but no model beats the equal-weight market in 2024. Quant-Arena is open to submissions from any model or agent harness.
Key facts
- Quant-Arena is a benchmark and public leaderboard for end-to-end quantitative research by LLMs on real A-share market data.
- A model gets feedback only on 2022 and 2023, then freezes one strategy that is assessed on the sealed year 2024.
- A cash-account evaluator applies trading constraints and costs, including market impact estimated from a simulated order book.
- Scores are dominated by realized excess return, complemented by signal quality and feature diversity.
- Nine frontier models were evaluated; GPT-6 Astra scores highest, yet no model beats the equal-weight market in 2024.
Why it matters
The authors say earlier benchmarks scored formulas, code, or single trading decisions, and so never answered whether an LLM can run the full research loop and produce strategies that hold up beyond the period they were developed on. Quant-Arena targets that gap with a sealed out-of-sample year. The headline result is sobering: the top-scoring model still fails to beat the equal-weight market in 2024.
Who it affects
Researchers building LLM agents for finance and anyone evaluating such agents are the direct audience. The leaderboard is open to submissions from any model or agent harness, so teams can enter their own systems. The data is A-share market data, so the results speak to that market.
How to use it
Teams can submit a model or agent harness to the public leaderboard. Under the protocol, the submission builds features and stock-scoring code, uses feedback from 2022 and 2023 only, and freezes one strategy for the sealed 2024 assessment.
How solid is it
This is a preprint, summarised here from its abstract. The design guards against overfitting to the research period by sealing 2024 and charging trading constraints, costs, and simulated market impact. The abstract does not give score values, returns, or excess-return figures for any model, and it does not name the other eight models, so the size of the gap to the equal-weight market is not stated.
Risks and caveats
The result covers one market (A-shares) and one sealed year (2024), and each model gets one frozen strategy to be judged on. The abstract does not say by how much the models trail the equal-weight market. It also says nothing about GPT-6 Astra beyond that it is the highest-scoring of the nine models, so treat that name with care until the paper's details are checked.