Glonce

DRACO turns one rubric score into per-step credit for AI agents

researchThe Glonce editorsHugging Face🔥79Glonce rating79Impact72Novelty78Relevance100Surprise60Players65Credibility85
DRACO turns one rubric score into per-step credit for AI agents

Reinforcement learning from verifiable rewards works when a task has a programmatic checker, but most long-horizon agent domains, such as multi-step app or tool use, have none. Researchers describe this as the outcome-blind setting, where ground-truth success signals are not available. Multi-criteria rubrics are a popular substitute reward, but they are scored only once per trajectory, and a single scalar judgment is a weak signal to spread across tens of steps of agent behavior. To address this, the researchers propose DRACO, short for Distributing Rubric-based Advantage for Credit Optimization. DRACO generates rubrics dynamically during training so they track the policy's evolving capability, scores those rubrics once per completed trajectory, and then redistributes that single judgment over the specific steps responsible for the rubrics that were annotated. This produces differentiated per-step advantages for use inside GRPO (Group Relative Policy Optimization). The redistribution itself is closed-form: it does not require training a separate attribution module. On the AppWorld benchmark, DRACO gains 15.9 points over the base model and 5.3 points over GRPO trained with a sparse ground-truth reward, despite not using any verifiers itself. Tested out-of-domain on Tau-Bench, DRACO gains 5.3 points over the base model even without a frontier judge, beating both ground-truth-reward training and other rubric-based training settings on that benchmark. The code for DRACO is available on GitHub at IBM/draco. The source text does not name individual authors or institutions, does not give training compute, model size, rubric-set size or typical trajectory length, and carries no publication or submission date.

Key facts

  • DRACO (Distributing Rubric-based Advantage for Credit Optimization) generates rubrics dynamically during RL training to track the policy's evolving capability.
  • It scores a rubric once per completed trajectory, then redistributes that single judgment over the responsible steps to produce differentiated per-step advantages inside GRPO, using a closed-form method with no trained attribution module.
  • On AppWorld, DRACO gains 15.9 points over the base model and 5.3 points over GRPO trained with a sparse ground-truth reward, without using any verifiers itself.
  • On the out-of-domain Tau-Bench benchmark, DRACO gains 5.3 points over the base model even without a frontier judge, beating both ground-truth-reward training and other rubric-based training settings.
  • Code is available at github.com/IBM/draco.

Why it matters

Reinforcement learning with verifiable rewards needs a programmatic checker, and most long-horizon agent tasks do not have one. Rubric-based rewards fill that gap but are usually scored only once per full trajectory, giving a single scalar to explain tens of steps of behavior. DRACO's closed-form redistribution turns that one trajectory-level score into per-step credit without training an extra attribution model, which targets exactly the outcome-blind settings where ground-truth signals do not exist.

Who it affects

The method targets researchers and engineers training long-horizon AI agents, such as app-automation or tool-use agents, with reinforcement learning in domains that lack an automated success checker and that already use GRPO-style training pipelines.

How to use it

The code for DRACO is published on GitHub at IBM/draco. It is designed to plug into GRPO training: rubrics are generated dynamically during training, scored once per trajectory, and redistributed into per-step advantages through a closed-form calculation, with no additional trained module required. The source text gives no pricing or licensing terms.

How solid is it

DRACO was evaluated on two agent benchmarks. On AppWorld, it gains 15.9 points over the base model and 5.3 points over GRPO trained with a sparse ground-truth reward, without using any verifiers itself. On the out-of-domain Tau-Bench benchmark, it gains 5.3 points over the base model even without a frontier judge, beating both ground-truth-reward training and other rubric-based training settings there as well. Testing across two benchmarks, including one out-of-domain, supports some generalization, though the source gives no further evaluation detail.

Risks and caveats

The source text does not name individual authors or institutions behind DRACO, gives no training compute, model size, rubric-set size, or typical trajectory length, and carries no publication or submission date. That leaves the cost and scale of training DRACO, and how far its gains generalize beyond the two benchmarks tested, unverifiable from the text itself.

“The redistribution is closed-form and does not introduce any trained attribution module.”

— the authors