Glonce

OpenFlyV2 puts a vision-language-action model on a quadrotor, 48% success zero-shot

researchThe Glonce editorsHugging Face🔥57Glonce rating57Impact42Novelty65Relevance92Surprise30Players8Credibility72
OpenFlyV2 puts a vision-language-action model on a quadrotor, 48% success zero-shot

The authors present OpenFlyV2, a single model that lets an aerial robot understand scenes, find things named in language, reason about space and fly, all without offboard computation. They frame the problem as one of organisation: human cognition draws on a shared perceptual foundation to understand environments, attend to referenced entities, reason about spatial relationships and act. Existing aerial systems, in their account, are often specialised for individual capabilities, dependent on offboard computation, or unable to sustain closed-loop operation on resource-constrained platforms.

OpenFlyV2 is described as a unified onboard vision-language-action architecture. A shared vision-language representation supports semantic understanding, language-grounded perception, spatial reasoning and continuous aerial control within one model. A compact action expert turns that shared multimodal context into short-horizon flight waypoints. Motion-aware temporal memory and action priors reduce the computational cost of closed-loop execution.

For training, the authors build a simulation-based dataset of approximately 70,000 navigation trajectories and 60,000 vision-language instruction-response pairs. (The dataset's name is garbled in the abstract.)

The model generates action chunks of 12 waypoints each, entirely onboard, at 10 Hz on an NVIDIA Jetson Orin NX. It transfers zero-shot from simulation to physical quadrotor flight, reaching a 48% navigation success rate across four real-world environments, with no real-world aerial training data. The authors conclude that broad vision-language competence and continuous closed-loop flight can coexist within a single model on a resource-constrained aerial platform. They say code will be made publicly available at https://github.com/Ideal-ljl/AeroBrain.

Key facts

  • OpenFlyV2 is one onboard vision-language-action model covering semantic understanding, language-grounded perception, spatial reasoning and continuous flight control.
  • It generates action chunks of 12 waypoints each, entirely onboard, at 10 Hz on an NVIDIA Jetson Orin NX.
  • Training uses a simulation-based dataset of approximately 70,000 navigation trajectories and 60,000 vision-language instruction-response pairs.
  • Zero-shot transfer from simulation to a physical quadrotor gave a 48% navigation success rate across four real-world environments, without real-world aerial training data.
  • The authors say code will be made public at github.com/Ideal-ljl/AeroBrain.

Why it matters

The authors argue that aerial robots usually split these abilities across separate systems, lean on offboard computation, or cannot keep a closed control loop running on small hardware. OpenFlyV2 is presented as a single model that handles language-grounded perception, spatial reasoning and flight control on the vehicle itself. The claim is that broad vision-language competence and continuous flight can share one model on a resource-constrained platform.

Who it affects

Mainly robotics researchers working on language-guided drone navigation and onboard embodied AI. The setup is a quadrotor with a Jetson Orin NX, so the work speaks to teams building small aerial platforms that cannot rely on a ground station for computation.

How to use it

Nothing is usable yet. The authors say code will be made publicly available at https://github.com/Ideal-ljl/AeroBrain, and no release date is given. Until then, the reported recipe is the takeaway: train on simulated trajectories and instruction-response pairs, then fly on real hardware without real-world aerial training data.

How solid is it

This is a preprint abstract from the authors, and all the claims are theirs. The 48% figure is an absolute navigation success rate across four real-world environments. The abstract gives no baseline or comparison, no per-environment breakdown, no number of trials and no definition of navigation success. It also does not state the model size, backbone, simulator or quadrotor model. The 10 Hz figure covers action-chunk generation on the Jetson Orin NX; no latency, power or memory figures are stated.

Risks and caveats

A 48% success rate means the system did not complete navigation in a little over half of the real-world trials, so it is a demonstration of feasibility rather than a reliable flight system. Without a baseline, it is hard to judge how that number compares with other approaches. Code is promised, not yet released, so independent checks are not possible for now.