Glonce

Self-hosted LLM absorbs 200+ enterprise apps via GRPO expert merge

researchThe Glonce editorsHugging Face🔥55Glonce rating55Impact70Novelty65Relevance95Surprise45Players30Credibility80
Self-hosted LLM absorbs 200+ enterprise apps via GRPO expert merge

A new paper describes an enterprise's attempt to stop its GPU pool from fragmenting: instead of running one self-hosted model per internal application, the team consolidated traffic from more than 200 internal applications onto a single model. The paper does not name the company, the model, or the baseline it compares against.

The problem is structural. Data-residency rules force many enterprises to self-host large language models rather than call an external API. But teams keep adopting newer models without retiring the older ones each app already depends on, so the serving fleet grows and a finite pool of GPUs gets split across more and more deployments. Consolidating traffic onto one model only works if that model can match the specialised ones it replaces, so the team ran production error analysis and grouped the gaps into three axes: instruction following, function-calling, and internal task distribution (a category the paper does not define further). Quality on each axis was tracked with offline benchmarks stratified to match real production traffic, scored either by deterministic verifiers or by calibrated LLM judges.

The team's first attempt, optimising all three axes jointly in one reward, ran into cross-domain reward interference: gains on one axis came at the cost of another. Their fix was to train a separate GRPO (Group Relative Policy Optimization) expert for each axis, then merge the three experts using a two-stage SLERP (spherical linear interpolation). Each single-axis reward turned out to expose its own failure mode when pushed too far: the instruction-following expert could collapse toward generic, semantically empty answers; the function-calling expert could over-call tools; the task-distribution expert could pad answers to game length-sensitive scoring. Each failure mode needed its own domain-specific fix before merging.

Measured in non-reasoning mode, the merged model beat a baseline that the paper describes only as about 7 times larger by total parameter count: 69.6 versus 65.8 on the team's in-house Arena, 0.85 versus 0.83 on instruction following, and 0.79 versus 0.77 on function-calling, while also lifting scores on general dialogue benchmarks. In production, the consolidated model now absorbs half of the self-hosting platform's total traffic, about 116 million requests a month, which the authors say it serves at a fraction of the prior cost, without giving a specific figure.

Key facts

  • Production traffic from more than 200 internal enterprise applications was consolidated onto a single self-hosted model.
  • Quality gaps were closed by training a separate GRPO expert per axis, one each for instruction following, function-calling, and internal task distribution, then merging the experts with two-stage SLERP.
  • In non-reasoning mode the merged model beats a baseline about 7 times larger by parameter count: 69.6 to 65.8 on the in-house Arena, 0.85 to 0.83 on instruction following, and 0.79 to 0.77 on function-calling.
  • The consolidated model now handles half of the self-hosting platform's traffic, roughly 116 million requests per month, at a fraction of the prior serving cost (no figure given).
  • The paper names neither the enterprise, the consolidated model, nor the baseline model it was compared against.

Why it matters

Enterprises that self-host large language models for data-residency reasons tend to accumulate one deployment per model generation, since retiring an old model means re-validating every application built on it. That practice fragments a GPU pool that does not grow as fast as the fleet does. This paper's answer is a concrete recipe for going the other way, folding many applications onto one model without giving up quality on any of them, by isolating each failure mode into its own reward and expert rather than trying to fix everything with a single blended objective.

Who it affects

The direct audience is post-training and MLOps teams inside enterprises that self-host LLMs across many internal tools and are running out of GPU capacity to keep adding models. It also speaks to teams doing multi-objective reinforcement learning from human or verifier feedback more broadly, since the cross-domain reward interference the paper describes, and the per-axis-expert-then-merge fix, generalises beyond the specific instruction-following, function-calling and task-distribution split used here.

How to use it

The reusable part of the recipe has three steps: run production error analysis to identify the specific axes where quality is failing, rather than optimising generically; train one GRPO expert per axis instead of one joint reward, since a shared objective let the axes fight each other; and merge the resulting experts with two-stage SLERP rather than fine-tuning further on a blended dataset. The paper does not name the model, the underlying architecture, or the company running it, so the recipe is not something a reader can download; it is a methodology to replicate against a team's own production traffic.

How solid is it

The evidence is a single, anonymised case study rather than a broadly reproducible benchmark: the paper names neither the company, the consolidated model, the baseline model beyond a roughly 7 times parameter-count comparison, nor the 'in-house Arena' and 'general dialogue benchmarks' used to score it. Quality was measured with offline benchmarks stratified to production traffic and scored by a mix of deterministic verifiers and calibrated LLM judges, which is a reasonable production-facing methodology, but external readers cannot inspect the benchmarks themselves. The traffic and cost figures, 50% of platform traffic and 116 million requests a month, are reported directly by the authors with no third-party verification available from the text.

Risks and caveats

The anonymity that makes the paper safe to publish also limits how much it can be checked: no company, model name, or timeframe is given, and the claimed serving-cost saving is described only as 'a fraction of the prior cost' with no number attached. The comparison baseline is defined solely by a relative size, about 7 times larger by parameter count, with no name or provenance, so readers cannot judge whether it was a strong or a weak baseline to beat. The recipe was validated on one enterprise's own traffic mix and internal benchmarks; how well the same three-axis split and SLERP merge generalise to a different application mix is untested in the text.