GST-Bench finds VLMs score 42.68 against humans' 79.08 on spatial reasoning
Researchers have introduced GST-Bench, a visual question-answering benchmark built to test global spatial intelligence in video understanding rather than the narrower local spatial perception that existing benchmarks focus on. Its questions are human-verified and derived from 6,790 minutes of synthetically generated video. To answer them, a model has to perform accurate spatial inference from viewpoints it never saw in the input video, and map what it observed from a first-person, egocentric view onto a global top-down image of the scene. The team evaluated 22 state-of-the-art VLMs on GST-Bench in a zero-shot setting. The strongest model reached a score of only 42.68, far below the human score of 79.08 on the same test. To find out what was driving the gap, the researchers built a second, narrower benchmark, GST-Bench-Local, that isolates local spatial understanding under the same task format. Models performed well there, showing they are competent at reasoning about nearby, immediately visible spatial relations. The gap only opens up when a model has to consolidate a long sequence of observations, gathered over time and from shifting viewpoints, into one globally consistent picture of the scene, which is exactly what current models fail to do. Alongside the evaluation benchmark, the team also released GST-Train, a dataset intended to support future work on global spatial reasoning.
Key facts
- GST-Bench is a VQA benchmark for global spatial intelligence in video, built from human-verified questions derived from 6,790 minutes of synthetically generated video.
- It requires models to infer spatial relations from viewpoints not seen in the input video, and to map egocentric observations onto a global top-down image.
- Across 22 state-of-the-art VLMs evaluated zero-shot, the strongest model scored only 42.68, against a human score of 79.08.
- A follow-up benchmark, GST-Bench-Local, shows models handle local spatial relations well under the same task format, but still fail to consolidate long-horizon observations into one globally consistent scene.
- The team also released GST-Train, a companion dataset meant to support future research on global spatial reasoning.
Why it matters
Spatial intelligence, being able to track where things are and how they relate to each other in a space, is treated as a foundational capability for embodied agents such as robots or navigation systems. Existing benchmarks have mostly tested local spatial perception from a single viewpoint or a handful of them, which leaves untested the harder skill of holding a consistent, global picture of a whole environment together as a model gathers observations over time from a moving camera. GST-Bench targets that gap directly, and the size of the gap it finds, a model score of 42.68 against a human score of 79.08, suggests today's leading VLMs are far from reliable at it.
Who it affects
The benchmark is aimed at researchers and teams building or evaluating vision-language models for embodied and video-understanding tasks, where an agent has to act on a spatial model of its surroundings built from a moving, first-person camera rather than from a single fixed image.
How to use it
GST-Bench is presented as an evaluation benchmark for testing a VLM's global spatial reasoning from video. The team also released GST-Train, described as a complementary dataset meant to support future work on the problem. The source does not give pricing, licensing terms or a release venue for either resource.
How solid is it
The benchmark's questions are human-verified rather than generated and left unchecked, and the underlying video corpus is large, 6,790 minutes. The headline result rests on a comparison across 22 separate state-of-the-art VLMs run zero-shot, not a single model, and the authors ran a dedicated diagnostic, GST-Bench-Local, specifically to isolate whether the gap comes from weak local spatial perception or from a failure to consolidate observations globally; they report it is the latter.
Risks and caveats
The source does not name the individual authors or institutions behind GST-Bench, give a release date or publication venue, identify which of the 22 evaluated models scored the top 42.68, or explain how the construction of GST-Train differs from that of the benchmark's own evaluation data. Because the video corpus behind GST-Bench is synthetically generated rather than filmed in the real world, how well the gap it measures predicts performance on real physical environments remains an open question the source does not address.