Synthetic Data-Based Training of Instance Segmentation: A Robotic Bin-Picking Pipeline for Chicken Fillets

Automating the manipulation of food objects is challenging due to their varying shape, size, and mass, coupled with deformability and varying orientation, making tasks such as bin-picking a difficult problem. We present a learning-based instance segmentation approach trained on synthetic data, and further fine-tuned using a limited set of real-world data. This approach allows to embed high object and scene variation in the training data, including accounting for deformability of the objects of interest. The result is a highly robust instance segmentation, which we combine with depth data to obtain a 3D spatial representation of the objects and environment. We complete the pipeline with grasp affordance and collision-aware path planning, and apply the approach to a chicken fillet bin-picking use-case as proof-of-concept. Validation of the instance segmentation model with real data yields a mask AP@50:5:95 of 0.68. Finetuning the model with a small dataset of real images increases the AP to 0.78. We experimentally validate the full bin-picking pipeline with a robot manipulator and mock-up chicken fillets. A supplementary video showcasing the pipeline is available online1.