SpatialBlock-15k trains vision-language models on 3D spatial reasoning

Large vision-language models (LVLMs) perform well on many visual tasks, but the authors say they remain limited at spatial intelligence: reconstructing and reasoning about the 3D structure of a scene from a flat 2D image. Existing fixes lean on real-scene spatial question-answering datasets that require dense geometric annotations, and building those labels is costly, time-consuming, and often noisy because it depends on external perception modules to generate them. The paper proposes a different route instead, inspired by how humans build spatial skills during cognitive development: teach a model foundational spatial reasoning through structured block-manipulation tasks rather than annotated real-world scenes.

To do that, the authors built SpatialBlock-15k, a synthetic dataset of 15,000 block-stacking problems spanning three task types: 3D-to-2D projection, viewpoint transformation, and structural combination. The dataset also adds controlled color modulation as a visual cue, meant to push models toward anchor-based reasoning (using color as a fixed reference point) even when a scene's block arrangement gets visually complex.

In experiments, LVLMs trained on SpatialBlock-15k, whether through direct answering or reasoning-based prediction, significantly outperform baseline models and generalize to real-world spatial tasks, despite the training data being synthetic and comparatively small. The abstract does not give an accuracy figure, name the baseline models, or describe which real-world tasks were used to test generalization. Code and the dataset are published on GitHub at rsoohyun/SpatialBlock; the paper's own text does not name an author or institution.

Key facts

SpatialBlock-15k is a synthetic dataset of 15,000 block-stacking problems covering three task types: 3D-to-2D projection, viewpoint transformation, and structural combination.
It replaces real-scene spatial datasets that need dense geometric annotations, which the authors call costly, time-consuming, and often noisy because they rely on external perception modules.
The dataset adds controlled color modulation as a visual cue to encourage anchor-based reasoning when a scene's block layout gets visually complex.
LVLMs trained on the dataset, via direct answering or reasoning-based prediction, significantly outperform baselines and generalize to real-world spatial tasks despite the training data's synthetic, compact nature.
Code and data are released on GitHub at rsoohyun/SpatialBlock; no author or institution is named in the paper's own text.

Why it matters

LVLMs are widely used for visual tasks, but reasoning about a scene's 3D structure from a single 2D image, spatial intelligence, has stayed a weak spot. The standard fix, real-scene datasets with dense geometric annotations, is expensive and often noisy because it depends on external perception modules to produce the labels. SpatialBlock-15k tests a cheaper, cleaner alternative: teach the underlying spatial skills through synthetic block-stacking puzzles instead, the way the authors say humans build spatial reasoning during cognitive development. The result's generalization claim stands out: a model trained purely on a compact, synthetic 15,000-example set reportedly still performs on real-world spatial tasks, the harder and more useful test of whether the approach works.

Who it affects

This affects researchers and engineers building or training vision-language models that need spatial understanding, especially teams currently paying for real-scene datasets with dense geometric annotations. The dataset and both training methods, direct answering and reasoning-based prediction, are released as a general resource for the LVLM research community, with code and data public on GitHub for others to build on.

How to use it

Code and the SpatialBlock-15k dataset are on GitHub at rsoohyun/SpatialBlock. The paper describes two ways to train on it, direct answering or reasoning-based prediction, both said to produce models that outperform baselines. Checking the repository directly is the way to see the exact setup and reproduce the training pipeline.

How solid is it

What is available here is the paper's own account of its experiments. The methodology is concrete: a defined dataset size of 15,000 problems, three named task types, an added color-modulation cue, and two distinct training strategies, both reportedly tested. Working against that: the abstract gives no accuracy, score, or percentage figure behind 'significantly outperform,' so the size of the improvement over baselines cannot be judged from the text. It also does not name the baseline or foundation LVLMs used for comparison, does not describe what the real-world spatial tasks used to test generalization actually are, and gives no date, publication venue, or timeframe for the work. There is no detail on how the 15,000 block-stacking problems were actually generated, such as which simulator or rendering pipeline was used, beyond calling the dataset synthetic.

Risks and caveats

Treat 'significantly outperform' as the authors' own characterization: the source attaches no number to it, so there is no way to check how large the gain over baselines actually is. The dataset itself is a synthetic world of colored blocks, a narrower and cleaner visual domain than most real photos, so how far its 3D-to-2D projection and viewpoint-transformation skills carry over to messier real scenes rests on the authors' own generalization tests rather than an external benchmark. No author names or institutional affiliations appear in the paper's own text; the linked code repository is hosted under the GitHub username 'rsoohyun,' which hints at, but does not confirm, an identity behind the work.

“We propose a novel paradigm inspired by human cognitive development: learning foundational spatial skills through structured block-manipulation tasks.”

— the paper