PAPER ANALYSIS
SynBalance (ICCV Workshops 2025) adds generated images for rare classes. On ImageNet-LT, accuracy on the rarest classes rises from 12.6% to 21.4%.
A vision model learns well what it sees all the time and poorly what it rarely sees. That’s class imbalance: some classes have thousands of examples, others almost none. SynBalance, presented at the ICCV 2025 workshops by researchers from AWS AI Labs and the University of Washington, tries to fix this by adding generated images to the rare classes. The numbers are good, but part of the gain comes from how the model is trained, not only from the images.
What SynBalance does
The paper studies long-tailed recognition: a few classes dominate the dataset. The authors analyze how four properties of synthetic images affect results: quantity, diversity, recognizability and domain gap. From that they build a three-part pipeline:
- Generation. Images from Stable Diffusion v1.5, with prompts shaped like “A photo of {a, several, a school of} {class} in {place}”.
- Composition. Instead of using everything, they pick a subset. Common classes get more diverse images; rare classes get the most recognizable ones.
- SynFusion. A two-stage training: first it mixes real and synthetic images (mixup); then it readjusts the classifier on real data only, with a loss that rebalances the classes.
What the results show
The numbers below are for ImageNet-LT with ResNet-50 trained from scratch (top-1 accuracy). Here "few" means classes with fewer than 20 real images, and "many" those with more than 100. Add-n means adding n synthetic images per class, with ordinary training.
| Setup | All | Many | Medium | Few |
|---|---|---|---|---|
| Real only (baseline) | 42.0 | 61.2 | 35.1 | 12.6 |
| Add-100 | 48.1 | 64.9 | 42.4 | 21.4 |
| Add-300 | 51.8 | 68.0 | 46.1 | 26.7 |
| Add-500 | 52.7 | 68.6 | 47.5 | 26.4 |
| Synthetic only, 100 per class | 18.9 | 19.4 | 18.7 | 18.3 |
- The gain is biggest where data is missing. With Add-100, “few” classes go from 12.6 to 21.4. “Many” classes rise less (61.2 to 64.9).
- Diminishing returns. From 100 to 500 images per class the overall score goes from 48.1 to 52.7, but “few” classes level off near 26.
- Synthetic alone isn’t enough. Training on synthetic only gives 18.9% at 100 per class and 28.1% at 500, well below the 42.0% baseline.
- Synthetic beats duplicating real. Duplicating real images of the rare classes (resampling) gave 41.3%, below the baseline. Adding synthetic images gave 48.1%.
Where the rest of the gain comes from. The paper’s final result (53.6% overall, 36.5% on "few" classes) adds two things. In the ablation, choosing the subset with their criterion adds 1.5 points over random picks, and SynFusion training adds 4.3. So the whole pipeline matters, not just the volume of images.
Against competitors. Among data-augmentation methods on ImageNet-LT, SynBalance edges out OTmix (53.6 vs 53.4), with 5.1 more points on "few" classes but fewer on "many" (62.7 vs 67.0). It isn’t the best overall: RIDE with 4 experts reaches 55.4. On iNaturalist, trained from scratch, SynBalance (69.2) fell below OTmix (71.1). The authors attribute this to Stable Diffusion 1.5 knowing those species poorly.
The limit: plausible isn’t the same as specified
- The generator caps fidelity. The authors themselves say relying on Stable Diffusion v1.5 limits image quality and the gain. Better models, such as DALL-E or Flux, could do more, at higher cost.
- You don’t specify the image. Diffusion returns a plausible image of the class, not the one you asked for in every detail.
- Bias risk. The paper notes that generation models carry their own biases, which can pass into the trained model.
- Scope. Public classification benchmarks, with a fixed training budget. Not a test on a real project’s data.
What changes with a controlled scene
Now picture the same problem with 3D simulation instead of diffusion. Rather than sampling images and filtering afterward, you declare the scene: lighting, materials, camera and how many examples of each class are explicit inputs. Here’s the contrast with full simulation versus generative AI. That gives you:
- Exact class balance, because the number of examples per class is an input
- Exact annotations, produced with the scene
- Reproducibility: the same parameters render the same dataset
Simulation can also carry bias, because we choose what to simulate, but it’s visible and auditable. The main point is the method: which classes to reinforce, how much realism, how much to mix with real, and how to train. The paper shows this in its own way, since selection and training counted almost as much as volume.
What to take to your project
- Reinforce what’s missing. The biggest gain came in the rare classes, not the common ones.
- Select and train with care. Choosing the images and adjusting training gave 5.5 more points than random picks with ordinary training.
- Validate with real data. Synthetic complemented real; alone it fell well short.
Dealing with a long-tail problem in your case? Tell us about it in a diagnostic call. If synthetic data isn’t the answer, we’ll say so.
Sources
- Jiang, Z., Cai, J., Liu, C., An, D., Wu, J. SynBalance: Harnessing Synthetic Data in Long-tailed Recognition. ICCV Workshops 2025, pp. 6335–6344.
Leave a Reply