Synthetic data in quality inspection

FUNDAMENTOS

In quality inspection, rare defects are the dataset problem. What an industry article says about synthetic data, and what still needs rigorous review.

In computer-vision quality inspection, the dataset tends to be smallest exactly where it matters most: rare defects. An article from A3 (the automation industry association) on vision AI for inspection describes the problem and brings the views of three vendors on the role of synthetic data. This post summarizes what the article says and doesn’t say, and where a controlled scene fits.

The problem: rare defect, small dataset

According to the article (John Lewis, A3, September 25, 2024), good manufacturers don’t produce many defects, so models need to be trainable with small datasets. Peter McLaughlin, of Agmanic Vision, calls getting a large enough, representative dataset, especially of defective parts, "the principal challenge," and says it’s often not possible to gather physical samples of every shape, size and color of defect.

This is where synthetic data comes in: generating images of parts and defects that real production rarely shows. The article gives one example, a classifier for welded parts leaving a paint booth, trained by Agmanic on images generated from CAD models, without capturing images on site.

What the article says about the limits

The article is a vendor-informed overview (Landing AI, Agmanic Vision and Invisible AI are quoted), with testimonials and examples, and it reports no measured results on the effect of synthetic data. What the interviewees say about the limits is consistent:

• It requires rigorous review. Quinn Killough, of Landing AI: synthetic data can help if a rigorous review process is used. AI-generated images may need human touch-up, which, per the article, “takes a lot of time and money.”
• It can worsen bias. McLaughlin: adding synthetic data, if not correctly applied, can exacerbate dataset bias.
• It reduces effort, it doesn’t remove real data. For him, synthetic data reduces the effort and time of collecting real images, but synthetic datasets must be well managed and, in some areas, still need real images.

The article’s only cost number (an 80% drop in labor costs at a semiconductor maker using Landing AI’s platform) is attributed mainly to streamlined workflows, not to synthetic data.

The limit

• An industry source, not a study. The article gathers vendor opinions and anecdotes. It is not an experiment or a numeric comparison between real and synthetic.
• This post’s old text promised more than the source says. Claiming that synthetic data “eliminates” collection costs or “has no privacy risks” doesn’t hold: modeling, render verification and review cost money, and licenses for third-party assets and models still apply.
• The render may not look like your line. If the scene doesn’t resemble what the camera sees in production, the model learns the wrong distribution.

What changes with a controlled scene

This section is our reading, not the article’s. In a 3D scene, the defect is a parameter: type, position, size and severity are declared, and the label (where it is, what it is) comes from the render itself. That lets you create the coverage real production doesn’t give and know what each image represents. The same caveats from the article apply to us: the scene needs review, the variation must reflect your line (shift lighting, part batch, camera angle), and the result is validated with real images, adjusting the scene for the failures that still appear. The dataset evolves with the model, in cycles, not as a one-off delivery.

What to take to your project

• Start with the defect that costs the most and count how many examples of it you really have today.
• Plan the review. Who checks the generated images, by what criteria, and how you detect bias the generator introduced.
• Validate on your line. Train, test on real production images, and go back to the scene for the cases that failed.

Does your inspection dataset cover the defects that cost the most? Tell us about it in a diagnostic call. If synthetic data isn’t the answer, we’ll say so.

Sources

• Lewis, J. Quality Inspection AI Vision: Synthetic Data Testing Training. A3 Automate, September 25, 2024. Industry article with vendor testimonials; treat as an indication, not a study.