Glonce

WROP dataset tests object permanence in 14 video world models

researchThe Glonce editorsHugging Face🔥59Glonce rating59Impact60Novelty70Relevance100Surprise50Players45Credibility85
WROP dataset tests object permanence in 14 video world models

Researchers introduce WROP (World Reasoning with Object Permanence), a data infrastructure built to test and train object permanence and solidity, cognitive traits humans develop early, in video generation models. The motivation is that video generation models, treated as a leading class of current world models, have started to show emergent reasoning abilities, which makes the question of whether they already grasp object permanence, and whether that trait can be trained in directly, worth answering. WROP consists of 150 hand-designed tasks inspired by cognitive science, split into six cognitive categories. The team built Blender-based generators that randomize nuisance factors such as speed, lighting, and camera angle while keeping each task's underlying cognitive structure intact, producing more than 10,000 samples per task. From this pipeline they release a 1.5-million-sample training corpus and a 300-question exam. On that exam they evaluate 14 video models spanning three architecture types: 3 reference-to-video models, 7 edit models, and 4 continuation models. Among the 14 is PWM-WROP, the authors' own 16-billion-parameter world model trained on the WROP corpus. In a blind pairwise Elo comparison, PWM-WROP ranks first among the continuation-model group and third overall, trailing only a statistical tie between two reference-to-video models. The authors release the training data, the exam, model answers and scores, PWM-WROP's weights, and PWM, their native PyTorch training stack, which runs on AWS Trainium2 hardware.

Key facts

  • WROP comprises 150 hand-designed cognitive-science-inspired tasks across six cognitive categories, generated via Blender to yield 10,000+ samples per task
  • The released training corpus contains 1.5 million samples, alongside a 300-question exam for evaluation
  • 14 video models were evaluated on the exam: 3 reference-to-video, 7 edit, and 4 continuation models
  • The authors' own 16B world model, PWM-WROP, trained on the corpus, ranks first among continuation models and third overall in a blind pairwise Elo study, behind a tie between two reference-to-video models
  • The authors release the data, exam, model answers, scores, model weights, and PWM, their native PyTorch training stack built on AWS Trainium2

Why it matters

Video generation models are increasingly treated as a form of world model, systems expected to internalize physical rules rather than just produce plausible pixels. Object permanence, the expectation that an object continues to exist when out of view, is one of the earliest cognitive priors humans form, and this work directly tests whether that prior emerges in video models or has to be trained in with a purpose-built dataset.

Who it affects

The release targets researchers and engineers building or evaluating video generation and world models, particularly teams working on physical reasoning, robotics simulation, or any application where a model's implicit physics matters for downstream use.

How to use it

The authors release the full 1.5M-sample training corpus, the 300-question exam, model answers and scores from the 14-model evaluation, the weights of their 16B PWM-WROP model, and PWM, their native PyTorch training stack that runs on AWS Trainium2, allowing others to reproduce the evaluation or train their own models on the same tasks.

How solid is it

The evaluation covers 14 models across three distinct architectural approaches (reference-to-video, edit, and continuation), and the headline comparison uses a blind pairwise Elo study rather than the authors simply grading their own model, which strengthens the result that PWM-WROP topped the continuation group and placed third overall.

Risks and caveats

The source material does not specify what the six cognitive categories individually test beyond the general framing of object permanence and solidity, and it gives no author names, institutional affiliations, or publication date, so independent verification of the claims rests on the released data and code rather than on the paper's provenance.

“In a blind pairwise Elo study, PWM-WROP ranks first among continuation models and third overall, behind only a statistical tie between two reference-to-video models.”

— the authors