Paper argues a trained agent's policy carries its harness's preferences

A paper titled "Your Policy Is Secretly a Harness: The Decision Geometry of Agent Interfaces", listed on Hugging Face papers, looks at how language-model agents are trained and judged. Such agents are post-trained and deployed through a harness: the prompts, tools and control logic that present a task to the model and execute its outputs. Harnesses are often revised and judged by how an already trained model performs under them. That practice assumes that what the model learned under one harness carries over to the next. The paper asks what a policy learns from its harness and what survives when the harness changes.

The central result is theoretical. Under KL-regularized control, the authors show that the log odds of an optimal policy recover how strongly its harness favors each action over the others, so a trained policy carries the preferences of its harness. Deployed return then splits into two terms. The ceiling of the harness is the best return it offers. The mismatch of the policy is the return lost to its own suboptimal decisions. Deployed return equals ceiling minus mismatch. Training changes only the mismatch, while changing the harness moves both terms. So comparing harnesses with a fixed model mixes what each harness offers with how well that model uses it.

The experiments start with MiniWoB++. There, post-training makes a model's preferences specific to its training harness. Reversing the order of the observation fields leaves every optimal decision unchanged, yet it lowers a trained model's success from 85.0% to 19.2%. The reverse also holds. Every new harness that a fixed-model comparison rejects, but that raises the ceiling, outperforms the original agent after further training under it. Ceiling and mismatch measured beforehand predict these gains.

On Terminal-Bench Lite, the paper looks at coding-agent harnesses. Ceiling minus mismatch, estimated with a stronger model as a stand-in for the optimum, reproduces the accuracy ranking of the harnesses. Either term alone does not. Code is available at https://github.com/2020-qqtcg/HRP.

Key facts

Under KL-regularized control, the paper shows that an optimal policy's log odds recover how strongly its harness favors each action, so a trained policy carries its harness's preferences.
Deployed return equals the harness ceiling (the best return it offers) minus the policy mismatch (return lost to its own suboptimal decisions). Training changes only the mismatch; changing the harness moves both.
On MiniWoB++, reversing the order of the observation fields leaves every optimal decision unchanged but lowers a trained model's success from 85.0% to 19.2%.
On MiniWoB++, every new harness that a fixed-model comparison rejects but that raises the ceiling outperforms the original agent after further training under it; ceiling and mismatch measured beforehand predict the gains.
On Terminal-Bench Lite coding-agent harnesses, ceiling minus mismatch, estimated with a stronger model as a stand-in for the optimum, reproduces the accuracy ranking; either term alone does not.

Why it matters

Harness changes are usually judged by running one already trained model under each candidate. The paper argues this mixes two things: what a harness offers and how well that particular model uses it. Its MiniWoB++ example makes the point sharply. A change to the order of observation fields that alters no optimal decision still dropped a trained model from 85.0% to 19.2% success. The paper also reports the opposite case: harnesses that a fixed-model comparison rejects but that raise the ceiling came out ahead after further training.

Who it affects

People who build and evaluate LLM agent harnesses, meaning the prompts, tools and control logic around a model, and anyone who post-trains agents and then revises the harness. The Terminal-Bench Lite result concerns coding-agent harnesses specifically.

How to use it

The paper offers a way to compare harnesses: estimate the ceiling and the mismatch, and use ceiling minus mismatch rather than a single fixed model's score. On Terminal-Bench Lite, that estimate used a stronger model as a stand-in for the optimum. The authors release code at https://github.com/2020-qqtcg/HRP.

How solid is it

This is a preprint listed on Hugging Face papers, and the text available is the abstract. The theoretical claim is derived under KL-regularized control. The empirical claims come from two benchmarks, MiniWoB++ and Terminal-Bench Lite. The abstract gives only one pair of success figures (85.0% and 19.2%); it gives no size for the MiniWoB++ gains and no Terminal-Bench Lite accuracy values, only that the ranking is reproduced. It does not name the trained models or the stronger stand-in model, or say how many harnesses were compared.

Risks and caveats

The abstract does not say whether the results hold outside KL-regularized control, MiniWoB++ and Terminal-Bench Lite. The 85.0% to 19.2% drop is a single trained model on one benchmark with one change (reversed observation-field order). On Terminal-Bench Lite the ceiling is estimated using a stronger model as a stand-in for the optimum, not measured directly.

“so a trained policy carries the preferences of its harness.”

— Paper abstract, Hugging Face papers