Glonce

Microsoft and Hugging Face train Qwen3.8-27B on database-check rewards

researchThe Glonce editorsHugging Face🔥61Glonce rating61Impact45Novelty60Relevance95Surprise35Players45Credibility75
Microsoft and Hugging Face train Qwen3.8-27B on database-check rewards

Microsoft and Hugging Face have published a follow-up to their first ThinkingBox post, this time about training rather than grading. ThinkingBox-Bench has 507 tasks. Each one gives an agent a real backend, a set of MCP tools and a simulated LLM user, then checks what actually changed in the database at the end, not which tool calls the agent made. The authors noticed that for most tasks such a check can return 1 or 0 straight from the final database state, so it can serve as a reward function as well as a grader.

They built a training pool from ThinkingBox-format tasks outside the benchmark and used the executable outcomes as rewards. The hard part was picking tasks worth training on, since most are either already solved or hopeless for the model at hand. The authors call their answer offline policy-relative difficulty: each task's success probability is estimated once with the base policy before training, and the selected pool stays fixed for the whole run. Re-probing difficulty during training is the path they say they did not take.

The reference run fine-tuned Qwen3.8-27B with full-parameter, token-mean GRPO on 187 tasks. Each update produced 18 groups of 8 rollouts, or 144 episodes, across 24 H100s. Training was synchronous with parameter and optimizer offload, AdamW used a 1e-6 learning rate with zero weight decay, and the run lasted 50 steps. Zero-variance groups were replaced DAPO-style. Backend timeouts were treated as system errors, not wrong answers, so affected groups were refilled instead of scored as failures. Every rollout is a full episode with a fresh backend, an isolated MCP session and GPT-5 Chat as the simulated user. Most rewards come from deterministic terminal-state checks; a minority of narrow rubric requirements can call a judge, also GPT-5 Chat. Training and evaluation share the code that interprets Qwen output (reasoning extraction, tool calls, argument typing, rendering, task state), so the reward measures the same agent that is evaluated.

Results are reported on the 507-task benchmark, with each task run 20 times. Three metrics matter: pass@1 is average success over all 10,140 attempts, pass@20 is the share of tasks solved at least once, and observed 20/20 is the share of tasks that pass every recorded attempt. After 50 steps, Qwen3.8-27B's pass@1 rose from 51.70% to 60.78%, and tasks passed on all twenty attempts rose from 38 (7.5%) to 107 (21.1%), almost three times as many. Pass@1 and observed 20/20 rose in all five domains. Pass@20 was mixed: up in neobank and consulting, down in retail, travel and auto insurance. Auto insurance had the largest pass@20 loss (7.0 points); retail had the largest observed-20/20 gain (24.5 points).

The authors read this as sharpening: repeatable success went up while at-least-once coverage slipped a little. One explanation they offer is that probability mass moved toward trajectories that work repeatedly while some low-probability paths disappeared. In their words, the model did not become broader; it became more repeatable.

One example: in a corporate hotel-booking task, a traveler asks for a room downgrade, but policy requires manager approval first. The base model changed the booking and issued refunds without approval in 15 of 20 attempts. The fine-tuned model preserved booking and payment state in all 20, and all expected records matched in 19.

The comparison with other models shows why the metrics diverge. From the first post: Kimi-K3 solved 93.89% of tasks at least once but passed only 68 of 507 (13.41%) on all twenty attempts. Claude Opus 5.5 beat Claude Opus 5 by 0.66 percentage points on pass@1, yet both passed exactly 241 tasks every time. On observed 20/20 the fine-tuned Qwen's 107 now sits above Kimi-K3 (68), and below GPT-5.4 (128), GPT-6 Astra (231) and Claude Opus 5.5 (241). GPT-5.6 Sol is slightly ahead on pass@1 (61.91% versus 60.78%) but passed fewer tasks every time (82 versus 107). Claude Sonnet 4.6 scored 59.19% on pass@1 and passed 102 tasks on every attempt.

Cost is priced the same way as in the first post: estimated cost of the full 507 x 20 campaign divided by tasks passing 20/20. For the fine-tuned model that comes to $8.00 per dependable task, close to GPT-5.4 ($6.80), GPT-6 Astra ($7.45) and Claude Opus 5.5 ($7.80). Estimated cost per successful attempt fell from $0.177 to $0.139. The authors stress these dollar figures are a comparative serving index, not invoices or cloud bills; they include model inference and exclude the training run and the policy probing used to curate the training set. The ThinkingBox-Training code is on GitHub.

Key facts

  • With 50 GRPO steps on 187 training tasks, Qwen3.8-27B's average success (pass@1) on the 507-task ThinkingBox-Bench rose from 51.70% to 60.78%.
  • Tasks passed on all twenty attempts rose from 38 (7.5%) to 107 (21.1%), while at-least-once coverage (pass@20) fell slightly and was mixed by domain.
  • No benchmark task was in the RL training pool; rewards come mostly from deterministic checks of the final database state, with a minority of narrow rubric items judged by GPT-5 Chat.
  • Training tasks were chosen once, offline, by estimating each task's success probability with the base policy; the pool stayed fixed during the run.
  • The reference run used 24 H100s, 18 groups of 8 rollouts per update (144 episodes), a 1e-6 learning rate and zero weight decay; the training code is on GitHub.

Why it matters

The post turns an evaluation tool into a training signal. If a task can be graded by checking the final database state, the same check can reward an agent for leaving the right records behind, whatever tool calls it took to get there. The reported result moves a 27B model on the reliability measure the authors care about: tasks passed on every one of twenty attempts nearly tripled, from 38 to 107, on tasks the model never trained on. That is still well below GPT-6 Astra (231) and Claude Opus 5.5 (241) on that count, but above GPT-5.6 Sol (82) and Kimi-K3 (68). The authors also argue that pass@1 hides this: GPT-5.6 Sol edges the fine-tuned model on pass@1 (61.91% versus 60.78%) yet passes fewer tasks every time.

Who it affects

Teams building agents that write to production databases, and anyone choosing a model for that work: the authors say pass@1 should not be read alone, because observed 20/20 is closer to what production reliability requires. It also concerns practitioners doing agent post-training who need a verifiable reward and a way to pick tasks that are neither already solved nor hopeless for their model. The released code is aimed at those readers.

How to use it

ThinkingBox-Training is available on GitHub. The reference recipe: full-parameter GRPO with token-mean loss on a fixed pool of 187 tasks chosen by offline policy-relative difficulty, 18 groups of 8 rollouts per update across 24 H100s, synchronous training with parameter and optimizer offload, AdamW at 1e-6 with zero weight decay, 50 steps. Each rollout gets a fresh backend, an isolated MCP session and GPT-5 Chat as the simulated user. Training and evaluation should interpret Qwen output identically; the shared code is in trainer/core/tb_interpret.py. Zero-variance groups were replaced DAPO-style, and groups hit by backend timeouts were refilled instead of scored as failures.

How solid is it

The evidence is a joint Microsoft and Hugging Face blog from the benchmark's own authors, with a held-out evaluation: no ThinkingBox-Bench task was in the training pool, and the benchmark's domains, databases, policies and most tools were not seen in training. Each task was run 20 times, so the pass@1 figure rests on 10,140 attempts. The authors note limits themselves: the two sets share the ThinkingBox framework, simulated-user machinery, terminal-state grading, a few common tools and a similar task style, and these shared elements limit the scope of the transfer claim. No third-party replication is mentioned. The result is for one model and one 50-step run.

Risks and caveats

Coverage did not grow. Pass@20 declined in retail, travel and auto insurance (auto insurance by 7.0 points), and the authors describe the gain as sharpening, not broader capability. Task selection used the base policy once and was never refreshed, and the authors note that difficulty is policy-relative. The cost figures are a comparative serving index, not bills, and leave out training and the policy probing used to curate the dataset. Some rewards depend on a GPT-5 Chat judge, and the simulated user is also GPT-5 Chat.

“Overall, the model did not become broader. It became more repeatable.”

— The authors, Microsoft and Hugging Face, ThinkingBox-Training blog post