Elastic Latent Recurrence removes the one-loop tax in looped language models

A latent recurrent language model reuses a shared block of layers over and over, so a single checkpoint can trade inference compute for accuracy simply by changing the loop count. The catch, according to the paper's abstract, is that at one loop such models typically fall well short of a non-recurrent model of the same size. That one-loop tax makes a separate non-recurrent model the better choice whenever latency matters.

The paper presents Elastic Latent Recurrence (ELR), a recipe for training looped models without this tax. At one loop, an ELR model costs the same to serve as a non-recurrent model of the same size and, trained on the same data, reaches similar accuracy. Each additional loop then buys more accuracy for more inference compute.

Three choices make it work: a gated state update with a learned step size, loop counts mixed within each optimizer update, and conditioning each pass on its loop index and budget. The first two preserve one-loop accuracy; the third lowers loss at every additional loop.

The abstract also describes several ways to cut cost. Mixed depths can be introduced late in pretraining to limit the added training cost, and with zero-initialized conditioning added at the switch, this late schedule reaches lower loss than mixed-depth training from scratch. In MoE models, reusing first-loop expert routes on later loops cuts mixed-depth update time by 11.7% with no measurable change in accuracy. At inference, keeping only the last loop's attention KV cache holds attention KV memory at its one-loop level while retaining most of the gain from additional loops.

The reported results come from 4B dense and 30B-A3B MoE hybrid models. On a ten-task average, ELR at one loop is 0.52 points below its matched baseline on the 4B dense model and 1.04 points above it on the 30B-A3B MoE hybrid. Four loops add 7.25 points on the dense model and 4.75 points on the MoE hybrid.

Carried through post-training, the recipe yields Koi-4B, a single checkpoint. It scores an estimated 18 on the Artificial Analysis Intelligence Index at one loop, matching Qwen3.5-4B, and 29 at four loops.

Key facts

Elastic Latent Recurrence (ELR) is a training recipe for looped language models that aims to remove the one-loop tax: at one loop, such models usually fall well short of a non-recurrent model of the same size.
On a ten-task average, ELR at one loop is 0.52 points below its matched baseline on a 4B dense model and 1.04 points above on a 30B-A3B MoE hybrid; four loops add 7.25 and 4.75 points respectively.
Three design choices drive it: a gated state update with a learned step size, loop counts mixed within each optimizer update, and conditioning each pass on its loop index and budget.
In MoE models, reusing first-loop expert routes on later loops cuts mixed-depth update time by 11.7% with no measurable change in accuracy.
The post-trained Koi-4B checkpoint scores an estimated 18 on the Artificial Analysis Intelligence Index at one loop, matching Qwen3.5-4B, and 29 at four loops.

Why it matters

Looped models promise something fixed-depth models cannot: one checkpoint that spends more or less compute depending on the loop count. Until now, according to the authors, the price was weak one-loop accuracy, which made a separate non-recurrent model the better pick whenever latency matters. ELR claims to remove that price. At one loop it costs the same to serve as a non-recurrent model of the same size and reaches similar accuracy, and each extra loop then buys more accuracy for more inference compute. If the claim holds, one checkpoint could cover both the fast path and the high-accuracy path.

Who it affects

Teams that train and serve language models and currently keep separate models for latency-sensitive and quality-sensitive uses are the natural audience. So are researchers working on recurrent or looped architectures, for whom the paper offers a specific training recipe covering dense and MoE models, plus a post-trained example in Koi-4B.

How to use it

The abstract describes a training recipe, not a product. For a practitioner, the ingredients are a gated state update with a learned step size, loop counts mixed within each optimizer update, and per-pass conditioning on loop index and budget. Mixed depths can be added late in pretraining to hold down training cost, with zero-initialized conditioning added at the switch. In MoE models, reusing first-loop expert routes on later loops shortens mixed-depth updates, and at inference, keeping only the last loop's attention KV cache holds KV memory at its one-loop level. The source does not state whether weights, code or a release of Koi-4B are available.

How solid is it

The source is an abstract only, and every number is the authors' own report. The text names no authors or institutions. The ten-task average is not broken down by task, and the baselines are described only as matched. The Koi-4B figure of 18 at one loop is given as an estimate, and the source does not say who produced it. The 29 at four loops is not marked as estimated in the same sentence. The comparison with Qwen3.5-4B is a match at the one-loop level, with no exact Qwen score given.

Risks and caveats

One loop is not a free win everywhere: on the 4B dense model ELR sits 0.52 points below its matched baseline. More loops mean more inference compute, and the source does not give the compute or latency cost of four loops in numbers. The KV-cache trick retains most of the gain from additional loops, not all of it. The 11.7% figure is a cut in mixed-depth update time during training, not an inference speedup. The source does not say how many loops beyond four were tested or how the gains scale with more loops, and it does not say whether the 7.25 and 4.75 point gains are measured against the one-loop ELR model or the matched baseline.

“Each additional loop then buys more accuracy for more inference compute.”

— Paper abstract