Bias in synthetic data: what a 2024 report says

ArquivoArchive

CategoriasCategories

Posts recentesRecent Posts

TagsTags

4D reconstruction AI and the brain AI ethics AI infrastructure AI transparency BEDLAM bias ChatGPT class imbalance computer vision dataset augmentation depth fundamentals generative models ghost AI Infinigen introduction long-tail recognition MIT study monocular video object removal opinion paper analysis pre-training procedural generation quality inspection real-data limit representation learning representativeness robotics semantic segmentation sim-to-real SMPL SURREAL synthetic data synthetic humans synthetic images tech giants video

Redes sociaisSocial Links

Bileki

Atualizado em Updated

PAPER ANALYSIS

A 2024 report lists the risks and biases of synthetic data, with no experiments of its own. What it says, where it is weak, and how to audit data origin.

"Synthetic data has bias" is an easy sentence to agree with and a hard one to measure. The report by Hao et al. (arXiv 2401.01629, January 2024) gathers the risks in an organized way and works as a checklist. It measures nothing, and it’s good to know that before citing it.

What the report does

It’s a review by a team at Huazhong University of Science and Technology, posted on arXiv (version 1). It covers three blocks:

  • How data is generated: statistical methods (distributions, Monte Carlo, interpolation, density estimation) and deep learning (VAE, GAN, diffusion, LLMs).
  • Where it’s used: vision, audio, natural language and health, with examples such as PersonX (rendered in Unity), MedGAN and SynthIE.
  • What can go wrong: a list of limitations and risks, followed by recommendations.

What the report claims

  • Shortcomings of synthetic data, in seven items: distribution mismatch with real data, incomplete data, inaccurate data, insufficient noise, over-smoothing, neglect of temporal dynamics, and too little inconsistency (variety of sources and conditions).
  • Risks in use: weaker model generalization, ethical and social implications (fictional content and misinformation), vulnerability to adversarial attacks, and legal compliance difficulties, such as in finance.
  • Distribution and representativeness: datasets built without attention to demographic diversity (gender, age, race) can produce models that work worse for other groups, and generators trained on internet data can reproduce that source’s biases.
  • Recommendations: use more advanced generative models, bring in domain knowledge (computer graphics, physics, cognitive science) to gain realism, create industry standards, document the method and generation parameters, validate models on real data, and update the synthetic sets.

The limit: a review, not evidence

  • There’s no experiment. The report doesn’t measure how much bias each method carries or test any of the fixes it suggests. The statement that advanced generative models avoid distribution shift is a proposal, not a result.
  • Some examples aren’t about synthetic data. The GoArt art-filter case (which changed skin tone in a painting) is a style model, and the MIT Tiny Images set was collected from search engines and withdrawn for discriminatory content. The text itself says the problem “is not limited to synthetic datasets.” The examples show bias in AI and in real data; they don’t measure bias in synthetic data.
  • The text is general. The risks apply to statistics, GANs, diffusion and LLMs at once, and what holds for clinical text doesn’t necessarily hold for rendered images. The report doesn’t separate a model-generated image from an image rendered from a scene.
  • It’s version 1 of an arXiv preprint. I haven’t checked whether it was later published in a journal or conference.

What changes with a controlled scene

This section is our reading. Three of the report’s recommendations, documenting the method, validating on real data and using computer graphics and physics knowledge, describe practices of those who generate from scenes. In a 3D scene, decisions about who and what appears are recorded as parameters, and the label comes from the scene. That lets you audit the recipe, not just look at images.

It doesn’t remove bias. Whoever builds the scene chooses the models, populations, environments and proportions, and those choices can be biased. There’s also the distance between the render and the real camera, which only a test on real data measures. What a controlled scene offers is declared bias, which can be inspected and corrected, not absent bias.

To see why the mean hides what matters, our analysis of the 2017 segmentation paper has per-class results, with gains in some and losses in others.

What to take to your project

  • Ask who’s missing. Define beforehand which groups, conditions and classes the model must serve, and check the coverage of each.
  • Demand the recipe. How the dataset was generated, with which parameters and who reviewed the choices. Without that, there’s nothing to audit.
  • Validate on real data, by segment. A model serves equally well the groups the dataset said it covered only if the real test, split by group, shows it.

Does your dataset cover who it needs to cover? Tell us about it in a diagnostic call. If synthetic data isn’t the answer, we’ll say so.

Sources

,
, , , ,

Leave a Reply

Your email address will not be published. Required fields are marked *