Glonce

VisionWeave saves 43% of visual tokens on average in Qwen3.8-27B

researchThe Glonce editorsHugging Face🔥75Glonce rating75Impact55Novelty65Relevance100Surprise35Players15Credibility70
VisionWeave saves 43% of visual tokens on average in Qwen3.8-27B

Multimodal large language models encode images and video into dense, fixed-size patch tokens, which is costly. The paper behind VisionWeave starts from the observation that visual information is unevenly distributed: some regions need fine-grained detail, while others can be represented compactly. Plain downsampling throws detail away, and the authors say existing token pruning and adaptive approaches remain limited in content-adaptive granularity, task generalization, and integration with modern MLLMs and serving infrastructure.

Their answer is a native capability they call elastic visual representation weaving: the model itself learns, end to end, where to allocate visual representations and at what granularity. VisionWeave builds this into frontier-level MLLMs through large-scale training and has two components. A gated spatial pooler constructs coarse-grained representations alongside the native fine-grained ones within a shared MRoPE coordinate. A granularity router then learns how to allocate between them depending on content.

The authors say they validated the capability on Qwen3.5-4B through self-distillation alone, then scaled it to Qwen3.8-27B with over 30K A100 GPU-hours. On Qwen3.8-27B, VisionWeave adjusts token savings to the visual content, saving 43.0% of tokens on average while retaining 98.9% of native performance across eight benchmarks. For comparison, token pruning baselines with a fixed 50% savings target preserved only 88% of performance. The authors also say extensive evaluations confirm robust efficiency-quality trade-offs across diverse tasks, resolutions and video frames.

On serving, the paper reports that when deployed on the SGLang serving engine the method achieves a 2.3x throughput gain, while reducing mean TTFT (time to first token) by 54.4% and mean TPOT (time per output token) by 60.6%. The authors conclude that they believe the results position elastic visual weaving as a promising capability for next-generation multimodal models.

Key facts

  • VisionWeave pairs a gated spatial pooler (coarse and fine representations in a shared MRoPE coordinate) with a granularity router that learns content-adaptive allocation.
  • Validated on Qwen3.5-4B through self-distillation alone, then scaled to Qwen3.8-27B with over 30K A100 GPU-hours.
  • On Qwen3.8-27B it saves 43.0% of tokens on average and retains 98.9% of native performance across eight benchmarks, versus 88% for token pruning baselines at a fixed 50% savings target.
  • On the SGLang serving engine the method gives a 2.3x throughput gain, with mean TTFT down 54.4% and mean TPOT down 60.6%.

Why it matters

Visual tokens are a major cost of running multimodal models, and fixed-size patch encoding spends the same effort on empty and detailed regions alike. VisionWeave moves the decision about granularity inside the model, instead of bolting on a pruning step afterwards. The authors argue this is a native capability that frontier-level MLLMs should have, and report that it holds up at the 27B scale with large gains in serving speed.

Who it affects

Teams that train or serve multimodal LLMs, especially for image-heavy and video workloads, are the direct audience. The throughput and latency numbers are reported on the SGLang serving engine, so people running inference on that stack have the closest match. The base models in the paper are Qwen3.5-4B and Qwen3.8-27B.

How to use it

This is a training method, not a drop-in switch. The paper says the capability was validated on Qwen3.5-4B through self-distillation alone and scaled to Qwen3.8-27B using over 30K A100 GPU-hours, so adopting it means a training run of that kind. The abstract makes no statement about code, weights or model release.

How solid is it

The headline figures come from the authors' own abstract: 43.0% average token savings, 98.9% of native performance across eight benchmarks, and a 2.3x throughput gain on SGLang. The eight benchmarks and the token pruning baselines are not named, and the 98.9% and 88% figures are relative to native performance, not absolute scores. The 43.0% is an average, and how it is averaged across benchmarks is not specified. The abstract names no authors or institutions.

Risks and caveats

The comparison with pruning baselines is at a fixed 50% savings target, while VisionWeave's 43.0% is an adaptive average, so the two are not savings-matched. The hardware, batch settings and workload behind the SGLang throughput, TTFT and TPOT numbers are not stated, and those serving figures are not tied explicitly to a specific model size. Qwen3.5-4B validation results are not given numerically, and the closing claim that this is a promising capability for next-generation models is the authors' belief, not a demonstrated result.

“visual information is unevenly distributed: some regions require fine-grained detail, while others admit compact representations”

— VisionWeave paper abstract