Glonce

F2G-Pose predicts object pose, size and shape from one RGB-D frame at 28 FPS

researchThe Glonce editorsHugging Face🔥55Glonce rating55Impact42Novelty58Relevance95Surprise22Players8Credibility75
F2G-Pose predicts object pose, size and shape from one RGB-D frame at 28 FPS

A paper on Hugging Face introduces F2G-Pose, a framework for category-level RGB-D object pose estimation. The task is to recover an object's 3D rotation and translation without instance-specific CAD models or reference views. The paper calls it highly ambiguous because of intra-class variations, symmetries and partial observations.

The paper says current solutions struggle to balance reliability and efficiency. Methods built on intermediate structures are prone to error propagation, while diffusion models suffer from high inference latency and complex post-hoc candidate extraction.

F2G-Pose is the authors' answer: a real-time direct, single-pass framework that predicts category-level pose, metric size and camera-frame completed shape from a single RGB-D observation in one feed-forward pass. Under the hood, it lifts dense visual foundation features to partial point clouds, encodes local geometry with DGCNN tokenization, and integrates object-level structure with a Mixture-of-Experts Transformer. Predicting size and completing the shape in the camera frame at the same time is meant to encourage metric consistency through dense geometric supervision.

The training setup is narrow. The task-specific network is trained only on pose-annotated synthetic SOPE data, and its RADIO backbone is frozen and externally pretrained. With that, the authors report state-of-the-art accuracy on SOPE and ROPE while running at 28 FPS for pose-only inference. They also report strong camera-frame geometry on HouseCat6D and real-object transfer on HANDAL.

The paper adds analyses of latency, robustness, shape completion and GenPose++ candidate behavior, which it says show the efficiency and reliability of the method. Real-world manipulation experiments are said to demonstrate deployment potential under segmented RGB-D inputs. The project page, code and pretrained models are available at https://jrryzh.github.io/F2G-Pose/.

Key facts

  • F2G-Pose predicts category-level pose, metric size and camera-frame completed shape from a single RGB-D observation in one feed-forward pass.
  • The architecture lifts dense visual foundation features to partial point clouds, uses DGCNN tokenization for local geometry and a Mixture-of-Experts Transformer for object-level structure.
  • The authors report state-of-the-art accuracy on SOPE and ROPE at 28 FPS for pose-only inference.
  • The task-specific network is trained only on pose-annotated synthetic SOPE data; the RADIO backbone is frozen and externally pretrained.
  • The project page, code and pretrained models are released.

Why it matters

Category-level pose estimation has to cope with unseen object instances, symmetries and partial views. The paper argues that existing approaches trade reliability against efficiency: intermediate-structure methods propagate errors, and diffusion models are slow and need complex candidate extraction. F2G-Pose aims to avoid both problems by producing pose, size and shape in one feed-forward pass.

Who it affects

Mainly researchers working on object pose estimation and shape completion from RGB-D data. The paper also points to robotic manipulation, citing real-world experiments with segmented RGB-D inputs.

How to use it

The authors say the project page, code and pretrained models are available at https://jrryzh.github.io/F2G-Pose/. The paper describes pose-only inference at 28 FPS. The deployment setting it describes uses segmented RGB-D inputs.

How solid is it

The claims come from the authors' own abstract. They report state-of-the-art accuracy on SOPE and ROPE, plus results on HouseCat6D and HANDAL, and they back the efficiency claim with latency, robustness and shape-completion analyses. The abstract gives no accuracy figures or margins over prior methods. The 28 FPS figure is for pose-only inference; no hardware is stated, and no frame rate is given for the full pipeline with shape completion. No latency numbers for diffusion baselines or GenPose++ are given, so no speedup factor is stated.

Risks and caveats

The 28 FPS number covers pose-only inference, not the full pipeline with shape completion. The manipulation experiments are described only as demonstrating deployment potential under segmented RGB-D inputs; no robot, tasks or success rates are given. The network is trained only on synthetic SOPE data, and real-object transfer is reported on HANDAL. No model size or parameter counts are given.