Glonce

Marigold V2 improves monocular depth accuracy by 16-26%

researchThe Glonce editorsHugging Face🔥54Glonce rating54Impact60Novelty65Relevance95Surprise45Players55Credibility85
Marigold V2 improves monocular depth accuracy by 16-26%

Monocular depth estimation, reading distance from a single image, is a fundamental but highly ill-posed computer vision problem, with downstream applications in scene reconstruction, computational photography, and robotics. A paper revisits Marigold, a set of techniques for repurposing modern image generation and editing models built on the diffusion transformer (DiT) architecture into monocular depth estimators, and introduces Marigold V2. The authors note that despite the field's maturity, recent models still struggle to generalize to out-of-distribution inputs and to produce sharp, detailed depth maps; Marigold V2 targets both problems.

The new recipe runs single-step inference starting from pretrained multi-step flow-matching models, adding quantization where needed so the model stays cheap to run while preserving its capacity. Analyzing the artifacts produced by naive training, the authors identify two effective remedies: aligning the model's internal representations with semantic features extracted from ground-truth data, and a 2-stage fine-tuning protocol built around a novel Sinkhorn-based loss.

The result is crisper, cleaner depth maps that generalize well to out-of-distribution inputs, with a combined 16-26% improvement in AbsRel accuracy over the previous best on the KITTI and ETH3D benchmarks; the paper reports this as one range covering both benchmarks together, not a separate figure for each. Qualitatively, the authors say the model resolves fur, foliage, and hair-thin edges that eluded prior models. The same approach also reaches state-of-the-art results on other dense regression tasks, including surface normals estimation and intrinsic image decomposition.

The text does not name individual authors, research institutions, or the specific prior models that make up the previous-best baseline. It gives no absolute AbsRel values, only the percentage change, and states no publication date. Beyond a project website hosted as a Hugging Face Spaces demo, it says nothing about a code or model-weight release.

Key facts

  • Marigold V2 revisits Marigold, an approach that repurposes diffusion transformer (DiT) image generation and editing models into monocular depth estimators.
  • The recipe runs single-step inference from pretrained multi-step flow-matching models, with quantization where needed, to stay cheap to run while preserving model capacity.
  • Two remedies fix naive-training artifacts: aligning internal representations with ground-truth semantic features, and a 2-stage fine-tuning protocol built around a new Sinkhorn-based loss.
  • Marigold V2 posts a combined 16-26% improvement in AbsRel accuracy over the previous best on the KITTI and ETH3D benchmarks, and resolves fine details like fur, foliage, and hair-thin edges that eluded prior models.
  • The same recipe also reaches state-of-the-art results on other dense regression tasks, including surface normals estimation and intrinsic image decomposition.

Why it matters

Monocular depth estimation, reading distance from a single 2D image, feeds directly into scene reconstruction, computational photography, and robotics, but it stays what the authors call a highly ill-posed problem: even mature models struggle to generalize past their training distribution and to render sharp, detailed depth maps. Marigold V2 attacks both weaknesses at once. It keeps the earlier Marigold idea of repurposing diffusion transformer image generation and editing models for depth estimation, but compresses inference to a single step from pretrained multi-step flow-matching models and adds quantization, so the quality gains do not require a heavier model to run.

Who it affects

The paper speaks to computer vision researchers and engineers building depth estimation into scene reconstruction, computational photography, or robotics pipelines, the three downstream uses the text names directly. Because the same recipe generalizes to other dense regression problems, it also reaches teams working on surface normals estimation and intrinsic image decomposition, where the paper reports separate state-of-the-art results using the same underlying approach.

How to use it

The text does not describe pricing, a license, or a release of code or model weights. Only a project website is mentioned, hosted as a Hugging Face Spaces demo, and that is the entry point the source gives for trying Marigold V2 or checking its outputs. Anything beyond that demo, such as downloadable weights or a training codebase, is not addressed in the source.

How solid is it

The headline figure, a combined 16-26% improvement in AbsRel accuracy over the previous best, covers the KITTI and ETH3D benchmarks together as one range, not as two separate per-benchmark numbers, and the text gives no absolute before-and-after AbsRel values to check that range against. It also does not name which prior models make up the previous-best baseline, so the comparison cannot be checked against specific competing results from the text alone. The claims about resolving fur, foliage, and hair-thin edges, and about reaching state-of-the-art results on surface normals estimation and intrinsic image decomposition, are the authors' own characterizations.

Risks and caveats

The source names no individual authors or research institutions, so the work cannot be attributed to specific people or a lab from the material alone; a Hugging Face listing field naming a submitter is separate from the paper's actual authorship and is not used here. No publication, submission, or acceptance date appears either, so the paper's review status is unknown. Because the reported gain is one combined percentage range rather than per-benchmark absolute figures, the text does not show whether the improvement is even across both benchmarks or driven mostly by one of them.