simulation-synth-adhesive

Simulation-based Data Augmentation for the Quality Inspection of Structural Adhesive with Deep Learning

We train different state of the art object detection models to automate the quality inspection of an industrial structural adhesive application. We compare the models trained on different datasets to assess the impact of augmenting a small dataset of scarce real defect images with synthetic samples generated in a simulated environment.

We show that not only are the models trained only on synthetic data capable of generalizing well to real scenarios, but also that training YOLOv4 on the augmented dataset shows significant improvement over the models trained using only the small real dataset.

Generating Synthetic Defect Images with Simulation

The simplified simulation environment (first animation) is implemented in CoppeliaSim and loosely resembles the real structural adhesive application process (second image). In the third animation we see the output from the YOLOv4 model (512x512) trained on the augmented dataset (88 real / 4000 synthetic images), using as an input the feed from the simulation camera above the workstation. Two defect types can be generated and detected by the model, namely discontinuities in the adhesive bead, or excessive volume of adhesive (i.e., blob).

Comparing Results from Training with Real, Synthetic and Augmented Datasets

While the model trained purely on synthetic data can generalize and detect most defects in the real set, it performs worse than all other models in the study. Contrastingly, the model trained on the augmented dataset generally performs better than all alternatives, resulting in higher number of true detections and tighter bounding boxes.

Synthetic	Real	Augmented

Results on the Real Test Set with Data Augmentation

Results from the entire test set of real images with the YOLOv4 model (512x512) trained on the augmented dataset (88 real / 4000 synthetic images) with 0.9173 mAP@0.50.

Back to top