VeriPhy audits physical errors in AI video with typed evidence records

Researchers have presented VeriPhy, an auditable system for checking whether AI-generated video clips are physically plausible, arguing that a video can look smooth and fluent while still breaking basic physical rules, and that a single scalar quality score cannot say which rule was broken or when. VeriPhy works by first sending the prompt to a text-only planner, which compiles it into a set of typed physical obligations (specific things the clip must satisfy) and builds a statically validated execution plan before any frame of the video is even examined. Only after that plan exists does the system look at the actual footage: it runs a fixed set of frozen low-level expert tools, including segmentation and tracking, object counting, eleven typed physical measurements computed over the tracked objects, depth estimation, OCR, and audio-event detection, and only the calls the plan declared are allowed to run. Each of these tool calls produces an evidence record that carries its own provenance, meaning the system can always point to where a given measurement or observation came from. Typed resolvers then combine the usable evidence records into a three-valued verdict for each obligation: supported, contradicted, or unknown, presented to a user as plausible, implausible, or abstain, with the full evidence trail attached so any verdict can be traced back to what produced it. To evaluate VeriPhy, the authors built a corpus of 1,500 video clips with human-annotated flaw records that pinpoint real generation failures in terms of what the prompt required, where in the frame it failed, and when in the clip it failed. On a smaller 149-clip core subset of that corpus, containing 304 such human-annotated flaw records, VeriPhy correctly accounted for 228 of them. By comparison, a published question-decomposition evaluator, run on the same clips against the same claims, accounted for only 164. A third baseline, simply prompting the same underlying model backbone monolithically without VeriPhy's structured pipeline, reached 222, close to VeriPhy's 228. The authors are explicit that raw recall does not distinguish VeriPhy from this simpler monolithic baseline. What distinguishes it, they argue, is that every one of VeriPhy's decisions keeps its evidence record and provenance attached, making its reasoning traceable one verdict at a time, and that this structured output could itself serve as the interface through which a critic's verdict is fed back into a video generation system to improve it.

Key facts

VeriPhy compiles a video prompt into typed physical obligations and a validated execution plan before examining any frame of the generated clip.
It gathers evidence using frozen low-level experts: segmentation and tracking, counting, eleven typed physical measurements, depth, OCR, and audio-event detection.
Each tool call returns a provenance-carrying evidence record, resolved into a three-valued verdict: supported, contradicted, or unknown (plausible, implausible, or abstain).
On a 149-clip core with 304 human-annotated flaw records, VeriPhy accounted for 228, versus 164 for a published question-decomposition evaluator and 222 for a monolithic same-backbone baseline.
The authors say raw recall does not separate VeriPhy from the monolithic baseline; the distinguishing feature is that every verdict retains an auditable evidence trail.

Why it matters

Video generation models are increasingly judged by how convincing they look, but visual smoothness says nothing about whether objects fall, collide, or persist the way physics demands. VeriPhy targets that gap directly: instead of a single opaque quality score, it produces a structured, auditable verdict for each physical obligation a clip is supposed to satisfy, with the evidence that produced the verdict attached.

Who it affects

The approach is aimed at teams building or evaluating video generation and world models, where a scalar score cannot say which rule failed or where. Because VeriPhy's structured verdicts carry provenance, the authors also frame them as a possible feedback channel back into the generation system itself, which would matter to anyone building a training or refinement loop on top of a video model.

How to use it

VeriPhy is a text-only planner paired with a fixed toolbox of low-level experts (segmentation and tracking, counting, eleven typed physical measurements, depth, OCR, and audio-event detection). The planner first compiles a prompt into typed obligations and a statically validated plan, and only the tool calls declared in that plan are permitted to run against the observed footage, which keeps the evaluation traceable rather than free-form.

How solid is it

The evaluation used a 1,500-clip corpus of human-annotated flaw records and a 149-clip core subset carrying 304 such records. VeriPhy accounted for 228 of those 304, against 164 for a published question-decomposition evaluator run on the same clips and claims, and 222 for a monolithic prompting baseline using the same backbone. The authors themselves note that recall alone puts VeriPhy close to the monolithic baseline; the case for VeriPhy rests on the auditability of its evidence trail rather than on recall margin alone.

Risks and caveats

The source text does not name the authors, their institutions, a publication venue, or a submission date, nor does it identify the published question-decomposition evaluator used as a comparison baseline. It also does not state a timescale for deployment or say whether VeriPhy's code, weights, or the 1,500-clip corpus will be released publicly, so independent verification is not yet possible from what is stated here.

“Recall alone does not separate it from prompting the same backbone monolithically, which reaches 222; what separates them is that each decision retains its evidence record and provenance, making the traces auditable one verdict at a time and usable as the interface through which a critic verdict could be written back into generation.”

— the authors