Glonce

Anthropic's Jack Clark moves self-improving AI forecast to 2027 after TasteVal benchmark

researchThe Glonce editorsThe Decoder🔥63Glonce rating63Impact50Novelty55Relevance95Surprise50Players50Credibility55
Anthropic's Jack Clark moves self-improving AI forecast to 2027 after TasteVal benchmark

A new benchmark called TasteVal, from the research lab P-Zero Research, measures how well AI systems design experiments on their own and interpret the results. The lab calls this ability "research taste": picking the right experiments and drawing the right conclusions from them. According to Anthropic co-founder Jack Clark, who writes the Import AI newsletter, the capability has roughly doubled every three months since December 2025. Based on those numbers, Clark now believes recursive self-improvement (RSI), where AI systems develop their own successors, could become possible as early as 2027. He had previously expected 2028.

P-Zero Research defines research taste as three abilities: identifying which research questions are worth pursuing, designing experiments to answer them, and correctly interpreting the results. TasteVal tests only the last two. Choosing the research direction, often seen as the core of scientific intuition, is already made for the model. The benchmark has eight tasks, each with a fixed research goal from AI model development and a measurable score. They cover training data selection, pretraining and fine-tuning language models, training on human preferences, and defending against adversarial prompts. The model decides which experiment to run next and uses the results to guide its following step. To keep coding skill from skewing the outcome, a fixed coding agent runs all experiments on a single Nvidia H100 GPU. Each run ends after 40 GPU-hours of compute or 120 hours of wall-clock time.

The human baseline came from 24 experts who had recently worked at places like OpenAI, Google DeepMind, Nvidia and Microsoft Research. They tackled the same tasks, with at least two people on each one, and the best human attempt per task served as the benchmark.

To make research taste quantifiable, the lab turns it into compute efficiency. Experiment compute is scarce in AI research, so picking the right experiments and reading their results correctly means fewer runs and more progress on the same budget. The lab draws on its "AI Futures Model," which predicts that experiment compute becomes the biggest bottleneck once coding is automated. TasteVal therefore measures how much compute a model needs to match the experts' scores. A model that gets there with half the compute has twice the research taste by this definition, and P-Zero Research calls this ratio the compute multiplier.

The top model, Anthropic's Opus 5.5, reached a compute multiplier of 2.3, matching expert-level scores with less than half the compute. The statistical range runs from 1.15 to 4.37, so at the low end the model would barely lead. A single run cost roughly one-thirtieth of what an expert attempt cost on average. Across several model generations, the multiplier has roughly doubled every three months since December 2025, with the reported doubling time ranging from 1.7 to 5.0 months. In prior years, model research taste grew much more slowly; the curve steepened only around the turn of 2025 into 2026.

The source is careful about what this shows. The benchmark does not prove that AI systems are already improving themselves, since it captures only a narrow slice of scientific work. Whether the steep curve is a lasting trend cannot be determined from a single benchmark, and the authors admit their results may overstate "how quickly research taste is improving." The tasks can be checked quickly and cheaply, which is the kind of task AI labs find easiest to optimize for, and the human comparison group included no top-tier researchers.

Clark has long called creativity the key factor in how fast AI systems can improve themselves. Earlier this year he wrote that models had not shown this ability at any groundbreaking scale. He now points to progress in mathematical breakthroughs and to benchmarks like TasteVal as signs that this is changing. In a long essay in May he had been more cautious, putting the odds of a system that independently trains a stronger successor at roughly 60 percent by the end of 2028 and 30 percent by 2027. The Decoder notes that the forecast comes from someone who co-founded the company whose model tops the benchmark, which adds a layer of bias.

TasteVal speaks to an objection critics had raised. AI researcher Herbie Bradley pushed back on Clark's May essay, arguing that models could plausibly take over the work of junior researchers, but research taste, creativity and coherent long-term research planning were nowhere in sight. On planning and evaluating experiments, TasteVal pushes against that view, though it says nothing about choosing a long-term research agenda.

The practical gap is illustrated by a study led by a team at Fudan University, which analyzed the development of the team's own model. AI agents contributed up to 55 percent of methodology proposals, but humans still made over 85 percent of the final decisions. The authors noted it remains unclear how AI can generate diverse research directions and assess their value before results are in, which is the kind of decision TasteVal takes off the models' plate.

The benchmark lands in a debate the major labs started. In June, Anthropic wrote that recursive self-improvement has not been achieved yet but "could come sooner than most institutions are prepared for." Anthropic CEO Dario Amodei now calls for a controlled slowdown in AI development, while OpenAI acknowledges that fully autonomous RSI is not happening yet and says it should only be pursued if human oversight is guaranteed. If the trend holds up in other tests, the implications cut both ways: AI development would get faster and cheaper, but a growing share of the research would become harder for humans to verify, because models would be planning and evaluating their own experiments.

Key facts

  • Jack Clark, Anthropic co-founder and Import AI author, now thinks recursive self-improvement could become possible as early as 2027, up from his earlier expectation of 2028.
  • P-Zero Research's TasteVal has eight tasks and tests two of the three parts of 'research taste': designing experiments and interpreting results, not choosing which questions to pursue.
  • Anthropic's Opus 5.5 scored a compute multiplier of 2.3 against a baseline of 24 human experts (statistical range 1.15 to 4.37), at roughly one-thirtieth of the cost of an expert attempt.
  • The multiplier has roughly doubled every three months since December 2025, with a reported doubling time of 1.7 to 5.0 months.
  • The source stresses limits: a narrow slice of research, easy-to-optimize tasks, no top-tier researchers among the humans, and a forecaster tied to the company whose model leads.

Why it matters

TasteVal measures a capability skeptics had flagged as a barrier to a major AI breakthrough: choosing and judging experiments rather than solving well-defined math or coding problems. In May, researcher Herbie Bradley argued that research taste was nowhere in sight; this benchmark pushes against that view for planning and evaluating experiments. If the measured trend holds, AI development would get faster and cheaper, but more of the research would be harder for humans to verify. The story also shifts the debate from raw model performance to how much independent research work language models can take on.

Who it affects

AI labs and the researchers who work in them are the most direct audience, since the benchmark's tasks come from AI model development and its human baseline came from people who recently worked at OpenAI, Google DeepMind, Nvidia and Microsoft Research. Anyone following the policy debate over recursive self-improvement is also affected: Anthropic, whose CEO Dario Amodei now calls for a controlled slowdown, and OpenAI, which says RSI should only be pursued if human oversight is guaranteed, both frame the question in public.

How to use it

This is a forecast and a benchmark result, not a tool, so there is nothing to adopt. The useful move is to read the headline number with its context: a compute multiplier of 2.3 for Opus 5.5 means expert-level scores with less than half the compute, and the statistical range of 1.15 to 4.37 means the lead could be slim. Watch whether other benchmarks reproduce the roughly three-month doubling, since the source says a single benchmark cannot settle whether the trend lasts.

How solid is it

Moderate to weak as evidence for the 2027 date. The source says the benchmark does not prove AI systems are already improving themselves. The doubling time carries wide uncertainty (1.7 to 5.0 months), the authors admit their results may overstate how quickly research taste is improving, the tasks are quick and cheap to check and so are easy for labs to optimize for, and the human group had no top-tier researchers. The Decoder also points out that Clark co-founded the company whose model tops the benchmark, which adds bias to the prediction.

Risks and caveats

TasteVal skips the first of the three abilities, choosing which research questions are worth pursuing, so it says nothing about setting a long-term research agenda. The Fudan University study found humans still made over 85 percent of final decisions even as AI agents contributed up to 55 percent of methodology proposals. The 2027 estimate is a shift in belief from Clark; the only stated odds are his earlier May figures of roughly 60 percent by the end of 2028 and 30 percent by 2027. If models do end up planning and evaluating their own experiments, human verification of the research gets harder.

“could come sooner than most institutions are prepared for”

— Anthropic, on recursive self-improvement, as quoted by The Decoder (June statement)