Investor's Last Exam tests AI agents on equity research with frozen data

A paper introduces Investor's Last Exam (ILE), a benchmark for AI financial agents doing equity research. It was led by experienced buy-side investors and fund managers. The starting point is that financial agents have to form forecasts and investment judgments from the evidence available at decision time, and their predictions are then checked against what actually happened. The authors say existing benchmarks measure financial knowledge, research workflows or forecasting, but that evaluating the full research process requires a strictly frozen information set and professional evaluation criteria.

ILE has three components. The first is a versioned financial database spanning 3.43 million document versions and nine source categories, supplemented by structured market data, with validated timestamps governing retrieval and computation. The second is a task taxonomy organized around professional analyst deliverables. The third is an expert-led framework that links scoring criteria to sources, calculations and assumptions through financial dependency graphs.

The evaluation spans 86 assignments and jointly assesses research quality, forecast accuracy and simulated investment performance. GPT-6 Astra and Claude Opus 5.5 lead overall at 68.1 and 66.9 out of 100. Yet even Astra scores only 52.5 on the Forecast & Model component. The authors conclude that even state-of-the-art models fail to translate overall research capability into predictive performance.

The paper argues that ILE shifts the evaluation unit from isolated capabilities to the evidence-assumption-judgment chain. That, it says, lets researchers locate where analytical advantages fail to turn into forecasts and investment judgments, and gives a traceable, diagnostic foundation for training and evaluating financial agents.

Key facts

Investor's Last Exam (ILE) is an equity-research benchmark for AI financial agents, led by experienced buy-side investors and fund managers.
Its versioned financial database holds 3.43 million document versions across nine source categories, with validated timestamps governing retrieval and computation.
The evaluation covers 86 assignments and jointly assesses research quality, forecast accuracy and simulated investment performance.
GPT-6 Astra and Claude Opus 5.5 lead overall at 68.1 and 66.9 out of 100, but Astra scores only 52.5 on Forecast & Model.
The authors say strong overall research ability does not carry over into predictive performance, even for state-of-the-art models.

Why it matters

Financial agents are judged on whether their forecasts and investment calls hold up against later outcomes. The authors argue that existing benchmarks test financial knowledge, research workflows or forecasting separately, and that testing the whole research process needs a strictly frozen information set. ILE tries to do that, with timestamped document versions so that an agent only sees what was available at decision time. The headline finding is a gap: the top overall scores are 68.1 and 66.9 out of 100, while the leading model's Forecast & Model score is 52.5.

Who it affects

The paper is aimed at researchers who build and evaluate financial agents. The benchmark's task taxonomy is organized around professional analyst deliverables, and it was led by buy-side investors and fund managers, so the work also speaks to people who do equity research for a living.

How to use it

The authors present ILE as a diagnostic tool for training and evaluating financial agents. Because scoring criteria are linked to sources, calculations and assumptions through financial dependency graphs, a researcher can trace where in the evidence-assumption-judgment chain an agent's analysis stops turning into a good forecast or investment judgment.

How solid is it

This is a preprint abstract from the Hugging Face papers page, so the evidence is the authors' own summary of their results. The reported figures are specific: 86 assignments, 3.43 million document versions, nine source categories. The text gives overall scores for only two models, GPT-6 Astra and Claude Opus 5.5, and the Forecast & Model score for only Astra. The two overall scores are given in the order the models are named.

Risks and caveats

The source gives no scores for other models and no Forecast & Model score for Claude Opus 5.5. It states no actual simulated investment returns or forecast accuracy figures. It does not say whether the benchmark or its data is publicly released, and gives no cutoff date for the frozen information set. No author names or institutions are given. The conclusion that models fail to turn research ability into predictive performance rests on the authors' own benchmark.

“Our results show that even state-of-the-art models fail to translate overall research capability into predictive performance.”

— Authors of the Investor's Last Exam paper