Experience Orchestrator lifts simulated advisor contact by 32 points over a naive LLM agent

Key facts

• The Experience Orchestrator combines a Contextual Bandit, a PID controller and a POMDP belief tracker to govern multi-turn LLM-to-LLM conversations.
• Across 60,000 simulated financial-services conversations, EO lifted the high-intent advisor contact rate from 46.1% (naive LLM control) to 78.1%, a +32 percentage point gain.
• Contextual Bandit content-variant selection accounted for 97% of the between-factor variance in outcomes, pointing to the governance policy rather than starting conditions as the driver.
• The governance layer matters most for visitors with no natural inclination to convert; for visitors already near alignment, a naive LLM's empathetic defaults were largely sufficient.
• All results come from LLM-to-LLM simulation; the PID controller has not been calibrated against real human unpredictability, and the authors flag live-traffic validation as the next step.

Why it matters

Multi-agent LLM deployments increasingly put one agent in a conversation with another party, human or AI, whose objectives do not automatically align with its own. The paper's diagnosis is that without an explicit shared goal function, these conversations tend to collapse rather than converge: one side gives up, the other stops adapting, and nobody gets what they wanted. EO is an attempt to engineer that missing shared goal function directly into the system with control-theory tools, rather than relying on the LLM to improvise consistent, adaptive behavior on its own.

Who it affects

The work targets teams building conversational agents meant to move a resistant counterpart toward a concrete outcome: the paper's example is a financial-services site agent trying to get a visitor to contact a human advisor. The architecture is relevant more broadly to any multi-turn LLM agent deployment where the other party's incentives are not already aligned with the agent's goal.

How to use it

EO layers three mechanisms on top of an LLM agent: a Contextual Bandit selects which content variant to present, calibrated using real-world web analytics data; a PID controller enforces behavioral consistency through dynamic schema constraints; and a POMDP belief tracker maintains a probabilistic model of the visitor's intent as the conversation unfolds. The paper does not describe a released tool or product, so there is no deployment path, licence or pricing to report here. This is a research architecture tested in simulation.

How solid is it

The headline numbers, a 32 percentage point lift and a 97% variance share attributed to content-variant selection, come from a large simulation run of 60,000 conversations, which gives the comparison statistical weight within its own setup. But every one of those conversations was LLM-to-LLM: the 'visitor' was itself a simulated agent instructed to exhibit psychologically realistic resistance, not a real person. The paper reports no statistical significance test or confidence interval alongside the 78.1% vs. 46.1% or 97% figures, and gives no detail on how the visitor personas or their resistance were constructed.

Risks and caveats

The authors are explicit that all findings are conditional on LLM-to-LLM simulation and that the PID controller has not been calibrated against real human unpredictability. They describe validating EO on live traffic as the critical next step, which means the core claim, that this governance layer works, has not yet been tested against actual people. The paper also does not name any real company, product or deployment; the environment is described only as simulated financial services, and no authors, institutions or dates are given in the available text.

“the absence of a shared goal function produces not competition but collapse: the visitor capitulates, the site agent stops varying its approach, and the conversation terminates without achieving either agent's stated objective”

— the paper