Prior-fitted diffusion transformers do Thompson sampling in-context

Thompson sampling is a randomized strategy for optimizing unknown functions in a black-box manner. In continuous global optimization, the authors say it has largely been restricted to Gaussian process priors, because these are essentially the only class of priors for which computing the posterior distribution has been tractable in practice.

The paper introduces prior-fitted diffusion transformers. These are diffusion models that perform Thompson sampling in-context at inference time, after being pre-trained with the prior-fitted network paradigm.

To make the models usable, the authors develop a set of practical non-Gaussian priors for general black-box functions. These include priors with characteristics that cannot be achieved under Gaussianity, such as priors over unimodal functions.

The work also looks at scale. The authors quantify the tradeoff between model size and the number of pre-training iterations, and show that these models follow scaling laws akin to those observed in transformers defined over other modalities.

On results, the claim is that across a set of global optimization benchmarks, prior-fitted diffusion transformers with non-Gaussian priors can achieve improved performance compared to traditional pipelines built on Gaussian processes. The claim is worded as "can achieve", not as a consistent win.

Key facts

  • Prior-fitted diffusion transformers are diffusion models pre-trained with the prior-fitted network paradigm that perform Thompson sampling in-context at inference time.
  • Thompson sampling in continuous global optimization has largely been limited to Gaussian process priors, since those have been essentially the only tractable class for computing the posterior.
  • The authors build practical non-Gaussian priors for general black-box functions, including priors over unimodal functions, which Gaussianity cannot express.
  • The models follow scaling laws akin to those seen in transformers over other modalities, and the authors quantify the tradeoff between model size and number of pre-training iterations.
  • Across a set of global optimization benchmarks, the models with non-Gaussian priors can achieve improved performance compared to traditional Gaussian-process pipelines.

Why it matters

The paper targets a practical limit of Thompson sampling in black-box optimization: the method has largely been tied to Gaussian process priors because they are essentially the only class with a tractable posterior. By pre-training a diffusion transformer and sampling in-context at inference time, the authors aim to open the method to priors that Gaussian processes cannot express, such as priors over unimodal functions. The scaling-law result also suggests these models behave like transformers in other domains as they grow.

Who it affects

Mainly researchers and practitioners who optimize expensive or unknown functions in a black-box manner and currently rely on Gaussian-process pipelines. It is also of interest to people working on prior-fitted networks and diffusion models, since the paper combines the two.

How to use it

The paper describes a workflow rather than a product: pre-train a diffusion transformer with the prior-fitted network paradigm on a chosen non-Gaussian prior, then run Thompson sampling in-context at inference time. The authors supply a set of practical non-Gaussian priors for general black-box functions. No code or model release is mentioned.

How solid is it

This is an arXiv preprint, and the claims come from the authors themselves. The headline result is worded cautiously: the models "can achieve" improved performance over traditional Gaussian-process pipelines on a set of global optimization benchmarks. No numerical results, benchmark names, effect sizes or speedups are given in the text available here, and which Gaussian-process baselines were compared is not stated.

Risks and caveats

The improvement is stated as a possibility across a set of benchmarks, not as a guaranteed gain. No model sizes, numbers of pre-training iterations or compute budgets are given, so the cost of pre-training relative to Gaussian-process methods cannot be judged from this text. No limitations or cases where the method underperforms are mentioned.

“Thompson sampling has largely been restricted to Gaussian process priors”

— From the paper's abstract