New benchmark shows video models fake physics despite acing VBench
Researchers have introduced Principia, a benchmark built to test whether AI video generators actually obey the laws of physics rather than just looking plausible. The core problem it addresses: measuring absolute motion in generated video is unreliable, because frame rate, object scale and camera calibration are often ambiguous or missing entirely. Principia sidesteps this by checking relational consistency instead. When two objects in the same scene are governed by the same physical law, their motions must satisfy predictable relationships to each other, and those relationships hold regardless of calibration. The benchmark covers eight physical phenomena: gravity, restitution, friction, rotational inertia, projectile motion, momentum, pendulum motion and mass spring oscillation, spanning translational, rotational, collisional and oscillatory dynamics, using real world scenes filmed under controlled protocols. The team also built a calibration independent consistency score that measures physical violations directly in image space, without needing to know real world units. Applying this to thousands of generations from six state of the art video generators produced a stark result: not one model scored above 0.42 on Principia, even though all six scored around 0.8 on VBench, the video generation benchmark widely used today. The gap suggests VBench largely measures visual quality and coherence rather than physical correctness. The researchers also tested whether vision language models could at least detect these physics violations after the fact. The best performing model reached only 67% accuracy, and most models tested near chance level, meaning they could barely tell physically correct video from physically broken video.
Key facts
- Principia is a new benchmark that checks Newtonian physics in generated video through relational consistency between paired objects, avoiding the need for absolute calibration.
- It spans eight phenomena: gravity, restitution, friction, rotational inertia, projectile motion, momentum, pendulum motion and mass spring oscillation.
- Across thousands of generations, six state of the art video generators all scored around 0.8 on VBench but none exceeded 0.42 on Principia.
- Vision language models asked to detect physics violations topped out at 67% accuracy, with most performing near chance level.
Why it matters
VBench is one of the most widely cited benchmarks for judging video generation quality, and a score around 0.8 reads as strong performance. Principia shows that this score says little about whether the generated motion actually obeys physical law: the same six models that look good on VBench collapse to just over half that score once checked for relational physical consistency. That is a warning that visual polish and physical correctness are being conflated in how the field currently measures progress.
Who it affects
The finding is most relevant to teams building and evaluating video generation models, since it exposes a blind spot in the metric many of them rely on. It also matters for anyone considering AI generated video for uses where physical plausibility counts, such as simulation, training data for robotics, or any application where motion needs to be more than visually convincing.
How to use it
Principia is presented as a benchmark and a calibration independent consistency score, not a product or paid tool. Its practical use is as an additional evaluation layer alongside VBench: a team can score a video model on Principia to check whether the physical behaviour of paired objects in a scene is consistent, independent of frame rate or camera calibration issues.
How solid is it
The result rests on thousands of generations from six state of the art video generators and real world scenes recorded under controlled protocols, giving the comparison a concrete empirical base. The source text does not name the six video generators, the vision language models tested, the authors, their institutions, or the exact generation count behind 'thousands', so those specifics cannot be verified from the material at hand.
Risks and caveats
No explanation is given in the source for why models score highly on VBench yet poorly on Principia, so the mechanism behind the gap is not established, only the gap itself. The 67% accuracy figure and the near chance level performance describe vision language models trying to detect physics violations, which is a distinct task from generating physically correct video, and the two should not be conflated.