NEO-VLA drops the vision encoder for robot action policies

Vision-language-action (VLA) models, which turn images and instructions into robot actions, are typically built on one premise: perception is first compressed by visual encoders, and action reasoning begins only afterwards. The authors of a new paper question that design. Manipulation, they argue, often hinges on exactly the visual signals most vulnerable to this compression: local geometry, boundaries, depth and subtle state changes. That leads to their central question: should an action policy inherit a fixed visual endpoint, or directly access evolving visual computation?

Their answer is NEO-VLA, which they describe as a structurally encoder-free vision-language-action framework. Instead of a pretrained visual encoder, images are tokenized by lightweight patch convolutional layers. The resulting image tokens then evolve jointly with language throughout the backbone. A flow-matching action expert can therefore draw on instruction-conditioned visual representations at different depths, not only at the end.

The authors report a consistent pattern from this change. Native architectures outperform encoder-based counterparts under controlled comparisons. Full fine-tuning reduces the gap but does not erase it, and additional scale alone cannot recover it. Layer-wise probing adds a second finding: the action-critical signals often peak well before the final visual representation. Across diverse manipulation settings and distribution shifts, the authors say NEO-VLA consequently achieves stronger and more robust control.

Their conclusion is that these results suggest embodied control benefits from access to visual representations across the computational hierarchy, rather than a fixed encoder output.

Key facts

NEO-VLA is a structurally encoder-free vision-language-action framework: images are tokenized by lightweight patch convolutional layers, with no separate visual encoder.
Image tokens evolve jointly with language through the backbone, so a flow-matching action expert can draw on instruction-conditioned visual representations at different depths.
The authors report that native architectures outperform encoder-based counterparts under controlled comparisons; full fine-tuning narrows but does not close the gap, and more scale alone does not recover it.
Layer-wise probing shows action-critical signals often peak well before the final visual representation.
The authors report stronger and more robust control across diverse manipulation settings and distribution shifts.

Why it matters

Most VLA models compress vision first and reason about action afterwards. This paper challenges that default, arguing that manipulation depends on fine visual detail such as local geometry, boundaries, depth and subtle state changes, which an encoder's abstraction can lose. The probing result is the sharpest piece of evidence: action-critical signals often peak well before the final visual representation, which would mean a fixed encoder output throws away useful information.

Who it affects

Robotics and embodied-AI researchers who build or fine-tune VLA policies are the direct audience. The finding is about architecture, so it bears on anyone choosing between a pretrained visual encoder and a native design for manipulation tasks.

How to use it

The practical idea is to tokenize images with lightweight patch convolutional layers, let them evolve with language through the backbone, and give the flow-matching action expert access to visual representations at several depths. The text available here mentions no code, weights or release plans.

How solid is it

The claims are the authors' own, drawn from the paper's abstract. They say the comparisons are controlled and cover diverse manipulation settings and distribution shifts. The abstract gives no numeric results: no success rates, benchmark names, margins of improvement or model sizes. The encoder-based baselines are not named, so the size of the advantage cannot be judged from this text.

Risks and caveats

The authors frame the conclusion cautiously: the results suggest, rather than prove, that embodied control benefits from multi-depth visual access. Without named benchmarks, robots or a split between simulation and real-world tests, it is unclear how far the result generalises. Full fine-tuning narrows the gap, so the advantage may depend on how a model is trained.

“Full fine-tuning reduces but does not erase the gap, and additional scale alone cannot recover it.”

— From the paper's abstract