Piccolo decision models report top Decision Index score of 75.59

Piccolo decision models report top Decision Index score of 75.59

Piccolo is a family of multimodal decision models ranging from 0.8B to 27B parameters. The authors say it coevolves model capabilities and training data through what they call Agentic Data Engineering and recursive self-improvement (RSI). The models predict probabilities over candidate answers, and they are meant for general-purpose text and visual judgments, agent harnesses, data workflows and interactive games.

The headline result is on the Decision Index Vision Full leaderboard. Piccolo-27B tops it with a score of 75.59, while Piccolo-9B and Piccolo-4B score 68.35 and 67.46 respectively. The authors add that all three rank first on the Private benchmark in their respective model-size categories.

The process behind the models is agent-driven. Research agents analyze model behavior, construct training data, run experiments, and use the results to refine their next proposals. Semantic slices and a shared research memory connect these actions across the development of the whole model family. The training recipes comprise hundreds of thousands of examples, so data selection and feedback-driven refinement are central to the self-improvement loop.

From this large-scale RSI exploration the authors draw three insights they call counter-intuitive. First, backbone-recipe co-adaptation outweighs raw model recency: earlier backbones can surpass newer generations on visual tasks under identical recipes. Second, targeted data curation lets 4B to 9B models rival 27B baselines on specialized workflows. Third, continuous loss reduction can decouple from, and even degrade, discrete decision accuracy.

The work comes with released weights and tools from the Data-Juicer and AgentScope ecosystems. The authors say Piccolo provides an actionable blueprint for building frontier decision models and scaling data-driven RSI.

Key facts

  • Piccolo is a family of multimodal decision models from 0.8B to 27B parameters, trained by coevolving the model and its data through Agentic Data Engineering and recursive self-improvement.
  • Piccolo-27B scores 75.59 on the Decision Index Vision Full leaderboard and ranks first; Piccolo-9B scores 68.35 and Piccolo-4B scores 67.46.
  • The authors say all three models rank first on the Private benchmark in their respective model-size categories.
  • Three reported insights: earlier backbones can beat newer ones on visual tasks under identical recipes, targeted data curation lets 4B to 9B models rival 27B baselines on specialized workflows, and lower continuous loss can degrade discrete decision accuracy.
  • Weights and tools are released together with the Data-Juicer and AgentScope ecosystems.

Why it matters

The paper treats training data as something a model and its research agents improve together, rather than a fixed input. The reported results point to data curation and recipe fit mattering as much as model size or freshness: smaller 4B to 9B models are said to rival 27B baselines on specialized workflows, and an older backbone can beat a newer one on visual tasks when the recipe is identical. The third insight, that falling continuous loss can decouple from decision accuracy, is a caution about using loss as the guide for decision models.

Who it affects

Mainly teams building decision models and agent systems: the models target agent harnesses, data workflows, interactive games, and general text and visual judgments. Researchers working on data-centric training and recursive self-improvement are the other audience, since the authors frame the work as a blueprint for scaling data-driven RSI.

How to use it

The authors say weights and tools are released together with the Data-Juicer and AgentScope ecosystems, and they present Piccolo as an actionable blueprint for building frontier decision models. The abstract does not state the license or where the weights are hosted, so check the paper page and the linked projects before planning any use.

How solid is it

The numbers are the authors' own claims for their own models. The source gives scores for Piccolo-27B (75.59), Piccolo-9B (68.35) and Piccolo-4B (67.46) on the Decision Index Vision Full leaderboard, and says all three rank first on the Private benchmark in their size categories. The score scale of the Decision Index Vision Full leaderboard is not stated, nor the scores of competing models or baselines, and no margin over the second-placed model is given. The backbone models behind the insights are not named.

Risks and caveats

Without competitor scores or the margin over the runner-up, the size of the lead cannot be judged. The claim that 4B to 9B models rival 27B baselines is limited to specialized workflows; the abstract does not say it holds in general. Scores for the 0.8B and other smaller sizes are not given. The abstract also does not say how many agents or RSI iterations were run, or how much compute was used, so the cost of reproducing the approach is unknown.

“Continuous loss reduction can decouple from and even degrade discrete decision accuracy”

— Piccolo authors, paper abstract