Glonce

Paper proves Platonic Representation Hypothesis for JEPAs

researchThe Glonce editorsHugging Face🔥66Glonce rating66Impact55Novelty70Relevance100Surprise40Players15Credibility70
Paper proves Platonic Representation Hypothesis for JEPAs

The Platonic Representation Hypothesis (PRH) conjectures that models trained on different modalities converge on the same representation of the world as they scale. The paper's starting point is that, without a proof, the hypothesis offers neither a guarantee nor a recipe. The authors say they turn it into a theorem.

The first result is what they call a Platonic Representation Theorem. Joint-Embedding Predictive Architectures (JEPAs) trained separately on any number of modalities learn the same representation up to one rotation, provided two conditions hold: their positive pairs are two observations of one scene (the authors call these good views), and their misalignment is bounded otherwise.

The second result reframes language modelling. The authors show that next-token prediction is a JEPA with bad views, because its positive pairs are parts of a single document. They say this explains why language models align with vision only through scale.

Third, the authors validate the theory on data ranging from 145k captions to the texts of 940M web images. They report a scaling law of alignment that predicts unseen runs within seed noise. Good views need 42 times fewer training samples than next-token prediction on the same corpus to reach the same alignment. A 109M-parameter text JEPA aligns with the DINOv2 and CLIP image encoders better than every language model they evaluated, up to 72B parameters, and also better than CLIP's text encoder. Their verdict: next-token prediction is a remarkably inefficient route to platonic representations.

As a showcase, one rotation turns separately trained encoders into an image captioner, which scores CIDEr .838 on COCO. Even that rotation needs no pair. On the latest pair-free benchmark, the authors' text JEPA beats every zero-shot accuracy reported with language models up to 8B parameters, for example 64% against at most 56% top-5 on ImageNet-100.

The conclusion: positive views, and not scale alone, are the lever to build platonic representations.

Key facts

  • The authors claim a Platonic Representation Theorem: separately trained JEPAs on any number of modalities learn the same representation up to one rotation, if positive pairs are good views (two observations of one scene) and misalignment is otherwise bounded.
  • They argue next-token prediction is a JEPA with bad views (parts of a single document), which they say explains why language models align with vision only through scale.
  • Good views need 42 times fewer training samples than next-token prediction on the same corpus to reach the same alignment; the validation spans 145k captions to the texts of 940M web images.
  • A 109M-parameter text JEPA aligns with DINOv2 and CLIP image encoders better than every evaluated language model up to 72B parameters, and better than CLIP's text encoder.
  • On a pair-free benchmark the text JEPA scores 64% top-5 on ImageNet-100, against at most 56% for zero-shot language models up to 8B parameters.

Why it matters

The idea that models trained on different modalities end up with the same internal picture of the world is central to AI, but until now it was a conjecture. This paper claims a proof for one family of architectures, JEPAs, and attaches a condition to it: the training pairs must be good views of one scene. That shifts the question from "how much scale do we need" to "which training pairs do we use". The authors' own conclusion is that positive views, not scale alone, are the lever for building platonic representations.

Who it affects

Researchers working on multimodal alignment and self-supervised representation learning are the direct audience, since the paper offers both a theorem and a scaling law of alignment. It also concerns anyone who relies on large language models for cross-modal alignment: the authors argue next-token prediction is a remarkably inefficient route, and they point to a 109M-parameter text JEPA that beats language models up to 72B parameters on alignment with image encoders.

How to use it

The abstract gives a practical takeaway rather than a recipe: build positive pairs from two observations of the same scene instead of parts of one document. The authors show that one rotation can join separately trained encoders into an image captioner (CIDEr .838 on COCO), and that this rotation needs no paired data. Their scaling law of alignment predicts unseen runs within seed noise, so it could be used to forecast results before training.

How solid is it

The claims come from the authors' own abstract on a paper listing; this retelling rests on that abstract alone. The theory is said to be proven and then validated empirically across a data range from 145k captions to the texts of 940M web images, and the scaling law is reported to predict unseen runs within seed noise. No independent replication or peer review is mentioned. The source gives no details of the proof, no size for the misalignment bound, and no form or fitted parameters for the scaling law.

Risks and caveats

The theorem holds under stated conditions: good views and bounded misalignment. How often real data meets them is not described in the abstract. The comparisons are the authors' own, and the source does not name which language models were evaluated beyond the size ceilings of 72B in the alignment comparison and 8B in the zero-shot benchmark. The pair-free benchmark is not named, and the captioner's CIDEr score is given without a baseline, so it is hard to judge how strong it is. The 42-times figure applies to the same corpus and the same alignment target, not to model quality in general.

“Positive views, and not scale alone, are thus the lever to build platonic representations.”

— From the paper's abstract