EnvACE trains AI agents via internal world rehearsal, not live environments

Training a large language model agent to use tools over long sequences of steps normally means running it against a real or synthetic executable environment, which is expensive to build and verify, or against an external simulator that is hard to keep grounded in reality. A new method called EnvACE, described in a paper on Hugging Face, takes a different route: the policy alternates between acting and rehearsing. It first generates a tool call, then switches roles and plays the environment itself, producing the response that call would trigger, and conditions its next decision on that self-generated response. Both roles, the acting policy and the simulated environment, are optimized jointly and end to end using task-success rewards, with no external environment involved during this training loop. The authors argue that this world rehearsal process lets the policy internalize how actions relate to the environment's responses directly in its parameters, effectively building an agent world model that supports decision making on its own. They report that EnvACE performs strongly and transfers well across four benchmarks, BFCL-v4, tau^2-Bench, VitaBench and FinMCP-Bench, and that it outperforms baselines that instead scale up the external environment. Controlled studies described in the paper show the benefit of world rehearsal holding across different model scales. The method also carries over to test time: the internalized world model lets the agent privately rehearse an action before committing to it for real, yielding further gains under a moderate rehearsal budget without any additional interaction with the outside environment. The authors frame world rehearsal as a new way to scale agent training past the limits set by how much external environment access is available. Code for EnvACE is public on GitHub.

Key facts

EnvACE trains LLM agents by having the same policy alternate between generating a tool call and playing the role of the environment that responds to it, with both roles optimized jointly on task-success rewards.
The approach removes the need for real or synthesized executable environments and external simulators during training.
It is evaluated on four benchmarks: BFCL-v4, tau^2-Bench, VitaBench and FinMCP-Bench, where the authors report it outperforms environment-scaling baselines.
At test time, the trained model can privately rehearse an action before committing to it, producing further gains under a moderate rehearsal budget with no extra external interaction.
The code is publicly available on GitHub at github.com/Within-yao/EnvACE.

Why it matters

Long-horizon tool-use training for LLM agents is bottlenecked by environment access: building or verifying real executable environments is costly, and external simulators drift away from reality. EnvACE's pitch is that the policy can learn to simulate the environment's responses itself and fold that simulation into training end to end, which if it holds up removes one of the standing constraints on scaling agent training.

Who it affects

Researchers and engineers building tool-using, long-horizon LLM agents, and anyone currently paying the cost of constructing or maintaining executable training environments or external simulators for agentic reinforcement learning.

How to use it

The paper reports results on four existing agent benchmarks, BFCL-v4, tau^2-Bench, VitaBench and FinMCP-Bench, rather than a new one, and the authors have released the code publicly on GitHub (github.com/Within-yao/EnvACE) for others to apply the method.

How solid is it

The evidence is what a research paper offers: benchmark evaluation across four suites plus controlled studies showing the effect holds across model scales, and a claim of outperforming environment-scaling baselines. The publicly released code is a point in favor of reproducibility. The abstract itself does not state specific numeric results, accuracy figures, or score deltas for any of the four benchmarks, and it compares only against environment-scaling baselines, not against non-agentic or non-reinforcement-learning approaches.

Risks and caveats

The paper does not define or size what it calls a "moderate rehearsal budget" at test time, so how much extra compute the private-rehearsal gains cost is unclear from the abstract. The comparison set is limited to environment-scaling baselines, leaving open how EnvACE stacks up against other families of agent training methods.

“Through world rehearsal, the policy internalizes the relationship between actions and their environment responses in its parameters, yielding an agent world model that directly supports decision making.”

— the paper's authors