Researchers propose Infinite-Parameter LLM that learns weights live

The paper starts from a familiar premise: scaling laws say a language model gets more capable with more parameters and more training data, and Mixture-of-Experts architectures have exploited this by activating only a slice of a huge stored parameter bank per token. But that success rests on static pretraining data. A deployed model also faces live interaction, facts a user supplies, corrections they give, that would make it more useful but sits outside the training set. A conventional model cannot learn from this, because its weights are frozen after training; the workaround is to place that knowledge in the prompt, through retrieval or instruction, where it is re-read on every request and discarded once the request ends.

The authors propose writing that live data into the model's weights instead. Taking a cue from Mixture-of-Experts, they describe the Infinite-Parameter LLM: a compact hypernetwork that turns the data supplied at run time into a low-rank modulation of a shared base network, so the feed-forward weights are generated from live data rather than pulled from a fixed bank. Where earlier weight generators read the context once and then freeze, this one carries a Bayesian belief over the generator's latent code and updates it online as the session continues, so the effective weight is re-derived from that evolving belief rather than fixed after a single read.

The claimed payoff: the stored footprint stays fixed size, yet the weights the model can compile from it are effectively infinite. Carrying run-time knowledge in the weights rather than the prompt, the authors argue, amortizes it in compute, frees up the context window, persists across turns, and can generalize better than in-context use.

The paper, submitted 16 September 2026, specifies an evaluation protocol for testing this approach against in-context learning and retrieval, but the abstract page reports no results from actually running it, no parameter counts, compute costs, or latency figures for the hypernetwork or base network, and no mention of code, weights, or a dataset release. No author names or institutional affiliations appear on the abstract page either.

Key facts

The Infinite-Parameter LLM uses a compact hypernetwork to turn live run-time data into a low-rank modulation of a shared base network's feed-forward weights, instead of storing that knowledge in a fixed parameter bank or feeding it through the prompt.
It maintains a Bayesian belief over the generator's latent code and updates it online as a session proceeds, so weights are re-derived from that evolving belief rather than fixed after a single read of the context, unlike prior weight-generation approaches.
The authors argue that carrying run-time knowledge in the weights rather than the prompt amortizes it in compute, frees the context window, persists across turns, and can generalize better than in-context learning.
The paper specifies an evaluation protocol to test the approach against in-context learning and retrieval, but reports no experimental results, benchmark scores, parameter counts, or compute costs.
The paper was submitted on 16 September 2026, and its abstract page lists no author names, institutional affiliations, code, weights, or dataset release.

Why it matters

Frozen weights are a structural limit on deployed language models: whatever a user tells the model during a session, a correction, a new fact, a preference, has nowhere permanent to live except the prompt, where it gets re-read every request and vanishes when the request ends. The Infinite-Parameter LLM reframes that as a generation problem rather than a storage problem, using a hypernetwork to compile feed-forward weights from live data and a Bayesian belief that keeps updating as a session runs, rather than committing to whatever the context looked like on a single read. If it works, it targets a real gap between what a session teaches a model and what the model retains after that session's prompt is gone.

Who it affects

This is an early-stage architecture proposal aimed at researchers and engineers who build memory, personalization, or continual-learning systems for language models, not a released product or a company shipping a feature. Nothing here changes how anyone deploys a model today.

How to use it

There is nothing to use yet. The abstract page names no code release, no model weights, and no dataset, and no pricing or licensing terms are mentioned because none of that exists at this stage; the paper is a proposed architecture and an evaluation protocol, not a shipped artifact.

How solid is it

The paper specifies an evaluation protocol for comparing the approach against in-context learning and retrieval, but the abstract reports no results from running that protocol, no parameter counts, and no compute or latency figures for either the hypernetwork or the base network it modulates. No author names or institutional affiliations appear on the abstract page, so there is no way to weigh the claims against a track record. Submitted 16 September 2026, this reads as a proposal awaiting its own promised evaluation, not a validated result.

Risks and caveats

Every benefit claimed, compute amortization, freed context window, persistence across turns, better generalization than in-context use, is argued in the abstract rather than demonstrated with numbers. Bayesian online updating of a generator's latent code is also a harder engineering and stability problem than static weight generation, and the paper gives no indication of how it behaves over long sessions or adversarial input. Until the specified evaluation actually runs and the results are public, the architecture should be read as a hypothesis, not a working system.

“The stored footprint stays fixed, yet the weights the model can compile are effectively infinite.”

— the paper's abstract