DAViD from Microsoft: vision trained on synthetic only

ArquivoArchive

CategoriasCategories

Posts recentesRecent Posts

TagsTags

4D reconstruction AI and the brain AI ethics AI infrastructure AI transparency BEDLAM bias ChatGPT class imbalance computer vision dataset augmentation depth fundamentals generative models ghost AI Infinigen introduction long-tail recognition MIT study monocular video object removal opinion paper analysis pre-training procedural generation quality inspection real-data limit representation learning representativeness robotics semantic segmentation sim-to-real SMPL SURREAL synthetic data synthetic humans synthetic images tech giants video

Redes sociaisSocial Links

Bileki

Atualizado em Updated

PAPER ANALYSIS

Microsoft’s DAViD trains depth, normals and masks for people on 300K synthetic images. See the numbers, where it trails Sapiens-2B, and the limits.

DAViD (Saleh et al., Microsoft, 2025) trains vision models for people using only synthetic images: 300K images, with depth, surface-normal and foreground-mask labels. The paper compares the results with much larger models, and that comparison is what’s worth reading, including where DAViD trails.

What DAViD does

DAViD adapts the Dense Prediction Transformer (DPT) to three dense tasks on people: relative depth, surface normals and soft foreground segmentation. The models share one dataset and one architecture, changing only the number of output channels, and the paper tests three sizes (Small, Base and Large, the latter with 0.34 billion parameters).

The data comes from SynthHuman, a synthetic dataset created by the authors:

  • 300K images at 384×512, split equally between face, upper body and full body.
  • Exact labels on every image: depth, normals and a soft mask, including details like hair and clothing.
  • Training: 100 epochs per task on the 300K images, on a machine with 4 A100 GPUs. The ViT encoder always takes 384×384, and a light module (Resizer) handles other resolutions.
  • Generation cost: 72 hours of rendering on 300 machines with M60 GPUs, which the paper equates to about 2 weeks of one machine with 4 A100 GPUs.

What the results show

The models are evaluated on two real-image sets, Goliath and Hi4D. The paper compares mainly with the Sapiens family, which pre-trains on 300 million real images and, according to the authors, took 18 days on 1,024 A100 GPUs for the largest model (Sapiens-2B, 2.16 billion parameters).

  • Depth (Table 1): DAViD-Large (0.34B, 663 GFLOPs) has the lowest RMSE on Hi4D, 0.072, versus 0.095 for Sapiens-2B (8,709 GFLOPs). On Goliath it wins on the face (0.140 versus 0.158) and loses on upper body and full body (0.283 and 0.334 versus 0.204 and 0.266). Averaged over the four sets, Sapiens-2B has the lowest RMSE (0.170 versus 0.191) and DAViD-Large the lowest AbsRel (0.012 versus 0.015). The paper sums it up as DAViD-Large performing on par with Sapiens-2B while running about 16 times faster (measured in MACs), plus beating Depth-Pro. The authors cite about 48 frames per second on an A100.
  • Normals (Table 2): the result is more modest. DAViD-Large trails Sapiens-2B in mean angular error on all three Goliath subsets (for example, 17.15° versus 16.04° on the face) and on Hi4D (15.37° versus 12.14°). Against Sapiens-0.3B, of similar size, the result is split: DAViD-Large has lower mean error on the face (17.15° versus 18.86°) and full body (14.60° versus 15.72°) and higher on upper body (13.96° versus 12.54°) and on Hi4D (15.37° versus 15.04°). The paper’s text says “on par” with the largest models and attributes part of the error to noisy, coarse test ground truth, lacking detail such as the mouth interior and wrinkles.
  • Soft mask (Table 3): on PhotoMatte85, SAD 5.85 versus 13.94 for MODNet; on PPM-100, 78.17 versus 104.35. The authors note their task is not full matting and that they used these sets for lack of a better benchmark; one of the compared methods is optimized for real-time matting and is more efficient.
  • What matters in the data (Table 4): with the same Large model and rendered datasets of similar size, depth RMSE on Goliath was 0.495 with THuman2.1, 0.278 with RenderPeople and 0.253 with SynthHuman. So the comparison is between synthetic sources (the first two scan-based), not synthetic versus real. With 60K images the Base model is already reasonable (RMSE 0.324), and it improves with 150K (0.305) and 300K (0.278).

The limit

  • One domain: people (face, torso, body). Nothing here was tested on industrial parts, vehicles or agricultural scenes.
  • A small, specific test: Goliath has 4 subjects (2,272 test samples, in 3 subsets) and Hi4D has 1,195 images with 6 subjects. In Table 2 (normals), the Hi4D numbers for the other models were taken from the Sapiens paper.
  • Not “synthetic versus real”: Depth-Pro, for example, was trained on generic datasets that include synthetic human data (BEDLAM), as the paper itself notes. The comparison is with much costlier models, not with a model trained on real data only.
  • Normals trail the largest model, and the authors acknowledge that the test ground truth (Goliath and Hi4D) is noisy, which can hide differences.
  • Declared failures: extreme lighting and rare scales, such as a baby held in an adult’s hand, where the large hand is perceived as closer to the camera than the face.
  • Bias: the paper acknowledges the model may be less accurate for some demographic groups, that aspects of human diversity are not yet represented in the dataset, and that there may be unfairness not yet discovered.

What changes with a controlled scene

This section is our reading, not the paper’s. Depth, normals and mask come out of the render itself, at the level of detail of the mesh (hair and clothing included), without the imprecision of a scan or a manual annotation. That is what Table 4 suggests: label fidelity and sample diversity weigh more than volume. Along the same line, the authors stress that control over the data makes it easier to check privacy, copyright, licensing and consent, and to fix imbalances the evaluation reveals. The cost is that the gap to the real world remains, and the paper’s own failure cases are exactly what the scene recipe didn’t yet cover.

What to take to your project

  • Label quality and diversity matter more than volume. In the paper, 300K well-labeled renders beat similar-size datasets from other synthetic sources.
  • Read the table, not the headline. DAViD ties or wins on depth, trails Sapiens-2B on normals, and the cost advantage is the core of the comparison.
  • Measure on real images from your case and document the groups and situations covered. The paper’s limits (rare scale, extreme light, diversity) show where to look.

Dealing with a vision problem for people or another domain? Tell us about it in a diagnostic call. If synthetic data isn’t the answer, we’ll say so.

Sources

,
, , ,

Leave a Reply

Your email address will not be published. Required fields are marked *