OmniLatent generates part-aware 3D assets from one image
A paper titled OmniLatent proposes a unified latent modeling framework for 3D part decomposition, the task of splitting a 3D object into editable parts. The authors start from a complaint about current practice: existing workflows typically segment a completed mesh as a post-processing step, which makes the result sensitive to geometric artifacts, texture ambiguity and limited part-level annotations.
OmniLatent takes a different route. Instead of predicting part labels in isolation, it learns a shared latent space that jointly encodes geometry, deformation, appearance and binary part IDs. The idea, as the authors put it, is that data-rich geometric and visual priors can then support part learning, where data is scarce.
Given a single input image, the base model generates geometry, deformation, texture and surface part IDs in one pass. The output is a part-aware 3D asset produced without a separate segmentation stage.
The authors then tune the pretrained base model for two extensions. The first is a point-prompt adapter. It maps arbitrary 3D point prompts with binary part IDs into latent tokens, and is used for full segmentation, refinement and part subdivision. The second is an amodal part completion model. It takes partial surface masks and completes them into a coherent full-object decomposition. To do that it uses dynamic per-part token allocation, cross-attention to the given geometry latent, and alternating part-local and global attention.
On results, the authors say extensive experiments show that OmniLatent produces semantically meaningful part segmentations with precise boundaries, supports interactive part editing, and completes partial part observations while preserving global object consistency. These claims are stated qualitatively.
Key facts
- OmniLatent learns a shared latent space that jointly encodes geometry, deformation, appearance and binary part IDs.
- From a single input image, the base model generates geometry, deformation, texture and surface part IDs in one pass, with no separate segmentation stage.
- A point-prompt adapter turns 3D point prompts with binary part IDs into latent tokens for segmentation, refinement and part subdivision.
- An amodal part completion model fills partial surface masks into a full-object decomposition using per-part token allocation, cross-attention to the geometry latent, and alternating part-local and global attention.
- The authors report that experiments show precise part boundaries, interactive editing and consistent completion, without quantitative figures in the text.
Why it matters
Editable 3D assets depend on being able to split an object into parts. The authors say the usual approach, segmenting a finished mesh afterwards, is sensitive to geometric artifacts, texture ambiguity and limited part-level annotations. OmniLatent folds part structure into generation itself, and uses abundant geometric and visual data to help with part learning, where labeled data is scarce.
Who it affects
The work targets people who build and edit 3D assets and the researchers working on 3D generation and part segmentation. The source frames part decomposition as a central capability for editable asset creation.
How to use it
The text describes the method, not a product. It names three uses: generating a part-aware 3D asset from one image, prompting with 3D points to segment, refine or subdivide parts, and completing partial part masks into a full decomposition.
How solid is it
This is a research preprint summary, and the claims come from the authors. They say extensive experiments support their results, but the text gives them only in qualitative terms. The text names no benchmarks, metrics or baseline comparisons.
Risks and caveats
The results cannot be judged from the text alone, since no numbers or comparisons are given. The text does not say whether code or weights are available, and it does not name the pretrained model that the base model builds on.