Glonce

JIT-Agent generates agent harnesses on the fly, pushes DeepSeek past GPT-5.6

researchThe Glonce editorsHugging Face🔥58Glonce rating58Impact62Novelty78Relevance98Surprise58Players42Credibility82
JIT-Agent generates agent harnesses on the fly, pushes DeepSeek past GPT-5.6

A paper posted to Hugging Face argues that an AI agent's capability is not set by its underlying model alone. The agent harness, the layer that handles memory management, planning strategy, action protocol, and tool or skill orchestration around a model, can dominate the contribution of the foundation model itself. The authors say harness design today is still manual, built per task, and does not scale.

To address that, they present JIT-Agent, a harness intelligence model trained to synthesize task-adaptive agent harnesses on the fly for arbitrary off-the-shelf agentic LLMs. The paper formalizes the agent harness as a composable, machine-generatable artifact governed by a fixed four-module protocol. JIT-Agent is trained to do three things: customize a harness for the task at hand, repair a harness so it runs stably and reliably, and self-evolve by distilling performance signals from a growing archive of prior harness configurations.

In the reported results, equipping DeepSeek-V4-Flash with JIT-Agent as a harness helper lets it surpass GPT-5.6 by 9.1 points on the DeepSearchQA benchmark and by 4.3 points on OdysseyBench. GLM-5.2, described as already strong, gains up to 20.2 points when given JIT-Agent-built harnesses. Across the paper's controlled evaluations, JIT-Agent-generated harnesses are reported as performance-competitive with mature agent runtimes such as OpenCode and Claude Code, and JIT-Agent is said to consistently improve model families at multiple scales, including DeepSeek V4, Mimo-V2.5, and Qwen3.6.

The authors frame JIT-Agent as, to their knowledge, the first model purpose-built for just-in-time harness generation, and argue the work establishes harness intelligence as a trainable, transferable, and compounding dimension of agent capability that is separate from scaling up the underlying model.

Key facts

  • JIT-Agent is a harness intelligence model that synthesizes task-adaptive agent harnesses on the fly for any off-the-shelf agentic LLM, formalizing the harness as a composable artifact under a fixed four-module protocol.
  • It is trained to customize harnesses per task, repair them for stable execution, and self-evolve by learning from an expanding archive of prior harness configurations.
  • Equipped with JIT-Agent, DeepSeek-V4-Flash surpasses GPT-5.6 by 9.1 points on DeepSearchQA and by 4.3 points on OdysseyBench.
  • GLM-5.2 gains up to 20.2 points with JIT-Agent, and JIT-Agent-generated harnesses are reported as performance-competitive with established runtimes OpenCode and Claude Code.
  • JIT-Agent is reported to consistently improve the DeepSeek V4, Mimo-V2.5, and Qwen3.6 model families across scales; the authors call it the first model purpose-built for just-in-time harness generation.

Why it matters

The paper's central claim cuts against a common assumption in agent development: that swapping in a stronger foundation model is the main lever for better agent performance. The authors argue the harness around the model, its memory management, planning strategy, action protocol, and tool or skill orchestration, can dominate that contribution instead, and that building harnesses by hand for each task does not scale. JIT-Agent tries to turn harness construction itself into an automated, trainable skill rather than a one-off engineering exercise.

Who it affects

The work is aimed at people building or evaluating agentic systems on top of off-the-shelf agentic LLMs, including the DeepSeek V4, GLM, Mimo-V2.5, and Qwen3.6 families named in the paper. The comparison against OpenCode and Claude Code as reference agent runtimes also makes it relevant to anyone benchmarking harness or scaffolding choices rather than just model choice.

How to use it

The source text does not give a release date, a code repository, or any statement about public availability, so there is nothing to point to for trying JIT-Agent directly. What is described is the method: a four-module harness protocol that JIT-Agent uses to customize, repair, and self-evolve harnesses for a given agentic LLM and task.

How solid is it

The benchmark gains, DeepSeek-V4-Flash's 9.1 and 4.3 point margins over GPT-5.6, and GLM-5.2's gain of up to 20.2 points, come from the authors' own reported evaluations on DeepSearchQA and OdysseyBench. The claim that JIT-Agent-generated harnesses are performance-competitive with OpenCode and Claude Code is stated without accompanying absolute scores for either runtime, so the size of that gap cannot be checked from the source alone.

Risks and caveats

The source gives no author names or affiliated institutions, no methodology detail beyond the high-level description of customizing, repairing, and self-evolving harnesses, and no absolute benchmark figures for the OpenCode or Claude Code baselines used in the competitiveness comparison. All performance numbers are self-reported by the paper's authors, with no indication of independent replication.

“Harness design remains manual, task-specific, and fundamentally unscalable.”

— the paper's abstract