Glonce

RoTA-Bench: GPT-5.6-Sol solves only 17% of 3D rotation tasks

researchThe Glonce editorsHugging Face🔥59Glonce rating59Impact45Novelty55Relevance95Surprise40Players30Credibility70
RoTA-Bench: GPT-5.6-Sol solves only 17% of 3D rotation tasks

A paper on Hugging Face introduces RoTA-Bench, a benchmark built around one question: can vision-language models turn 2D observations into 3D understanding that supports precise action? Doing so requires inferring object geometry, anticipating what a rotation will do, and refining actions through visual feedback.

The task works like this. A model is given a target silhouette and sees a rotated 3D object. It then chooses successive rotations to recover the target projection within a limited action budget. The authors call this interactive 3D-to-2D projection alignment.

The tasks come from an automated pipeline. It constructs silhouette-preserving 3D objects from 2D icons and uses reversible initialization to guarantee that every task can be solved. The pipeline yields 674 tasks, plus a stratified 100-task test set.

The authors report three empirical findings. First, a substantial human-model gap persists: GPT-5.6-Sol succeeds on only 17% of tasks, whereas 20 undergraduate participants collectively complete the test set four times with a 100% success rate. Second, progress in image space and in pose space can diverge. At matched states, the model more often improves silhouette overlap while worsening pose alignment than humans do (19.4% vs. 10.9%). Third, fine-grained control is sensitive to interaction history. Removing the history leaves 96.8% of the model's actions at a coarse 30-degree step size and sharply reduces observed task success.

The authors conclude that the results highlight the challenge of converting visual evidence into precise 3D control, beyond recognizing or matching individual images.

Key facts

  • RoTA-Bench asks a model to choose successive rotations of a 3D object so that its projection matches a target silhouette within a limited action budget.
  • An automated pipeline builds objects from 2D icons with reversible initialization, which guarantees solvability, and yields 674 tasks and a stratified 100-task test set.
  • GPT-5.6-Sol succeeds on only 17% of tasks; 20 undergraduates collectively complete the test set four times with a 100% success rate.
  • At matched states, the model improves silhouette overlap while worsening pose alignment more often than humans do: 19.4% vs. 10.9%.
  • Without interaction history, 96.8% of the model's actions fall at a coarse 30-degree step size, and task success drops sharply.

Why it matters

Many benchmarks test whether a model can recognize or match images. This one tests whether it can act on what it sees in three dimensions, step by step, with feedback. The authors say the findings show how hard it is to convert visual evidence into precise 3D control, beyond recognizing or matching individual images. A 17% success rate for GPT-5.6-Sol against a perfect human result on the same test set makes the gap concrete.

Who it affects

The paper is aimed at people building and evaluating vision-language models, especially those who want models to act on visual input rather than only describe it. The source names only one model, GPT-5.6-Sol, so it says nothing about how other systems perform.

How to use it

RoTA-Bench offers 674 tasks and a stratified 100-task test set, all generated automatically from 2D icons. Its setup can serve as a probe of whether a model can infer geometry, anticipate rotation effects and correct itself from visual feedback. The source gives no release date for code or data.

How solid is it

These are the authors' own reported results, taken from the paper's abstract. The human baseline is small: 20 undergraduates who collectively complete the test set four times, all successfully. The source names only one model, GPT-5.6-Sol, and gives no scores for others. The tasks are solvable by construction because of the reversible initialization.

Risks and caveats

The size of the action budget is not stated. The source gives no number for success without interaction history, only that it drops sharply. It does not say whether the 17% is measured on the 100-task test set or on all 674 tasks, and it does not say what the 19.4% and 10.9% are percentages of beyond 'at matched states'. The comparison rests on one named model and a small group of undergraduate participants, so the results should be read as a first look rather than a ranking of models.

“These findings highlight the challenge of converting visual evidence into precise 3D control, beyond recognizing or matching individual images.”

— RoTA-Bench paper abstract