SynBalance: balancing rare classes with synthetic data

PAPER ANALYSIS

An ICCV 2025 paper uses generated images to repair long-tailed class imbalance — and gets real gains on the classes models usually ignore. It also walks straight into the limitation that matters most to us: diffusion generates plausible images, not specified ones.

Computer vision models tend to learn a lot about what they see often — and very little about what they see rarely. This is the long-tailed distribution problem: some classes have thousands of examples, while others appear a handful of times in an entire dataset. The model isn’t failing because it’s badly built. It’s failing because the data never insisted.

What SynBalance does

SynBalance is a pipeline that combines real and synthetic data to improve performance on unbalanced datasets. The idea is simple, and powerful: generate additional images only for the classes that lack examples — and do it intelligently, by adjusting not just the quantity but also the diversity and clarity of what gets generated.

The authors used diffusion models such as Stable Diffusion v1.5 to create new images from text prompts. Those generated samples are then analysed and selected against four criteria:

• Diversity — how different the synthetic images are from one another
• Recognisability — how easily a model can tell what it is looking at
• Domain gap — the visual distance between the synthetic and real samples
• Quantity — how many new samples each class actually needs

That adaptive selection process produces a “balanced” dataset mixing real and synthetic images. Across benchmarks such as ImageNet-LT and iNaturalist, the method outperformed several baselines and gave a measurable lift on long-tail classes — the ones models normally overlook.

But there’s an important catch

The authors are upfront about it. Their synthetic images came from Stable Diffusion v1.5 — a general-purpose generative model built for art and illustration, not for scientific or machine learning datasets.

So while SynBalance delivered real improvements, image quality and realism remained a constraint. Some generated samples lacked visual fidelity, which capped the benefit. And generative pipelines tend to be computationally expensive and unpredictable: you cannot fully specify what each image will contain.

The concept is excellent. The generation method still has room to improve — and that’s the interesting part.

What changes if the data comes from a controlled scene

Now imagine the same pipeline with simulation behind it instead of diffusion. Not a distribution to sample from, but a scene to declare — with lighting, materials, camera positions, object classes and their frequencies all as explicit inputs. That would give you:

• Photorealistic, internally consistent images without generation artifacts
• Class balance set exactly, because the number of examples per class is an input
• Annotations produced automatically and exactly — boxes, masks, instance labels, depth, even spectral bands
• Reproducibility: the same scene parameters render the same dataset twice

That is the difference between asking a model to imagine a tiger and building a tiger you can position, light and photograph on demand. It’s also the distinction SynBalance points at without being able to make it: the next step is from AI-generated to design-controlled.

What the paper really teaches

The clear message is that synthetic data works, and that used purposefully it’s a serious tool against the limits of real-world datasets. The nuance — and the one worth carrying into your own project — is that the value is not in the volume of generated images. It’s in whether you control what’s in them, and whether you can prove what’s in them.

That’s the difference between a synthetic image and a synthetic dataset.

Computer Vision · Synthetic Data · Paper analysis · Class imbalance