A Robotic Sorting Method Of Small Soft Packages Based On Deep Learning

Sorting operation is widely used in industrial production,waste disposal and logistics distribution.However,at present,most industries still use manual sorting,which has high repeatability and high intensity.Soft packages are common packaging goods in daily consumption.Soft packages are prone to deformation.The surface is usually reflective or transparent.Aiming at the above problems,this paper mainly studies the robotic sorting strategy,grasping positioning and recognition of soft packages.The specific work of this paper is as follows:Firstly,a two-stage sorting control strategy of "grasp first,look later" was proposed for the dense stacking mixed soft packages.That is grasping and positioning unrelated to the target object category was realized first,and then the soft packages was classified and identified.Secondly,according to the strategy of "grasp first,look later ",a soft packages positioning method based on affordance map is proposed.In this method,full convolutional neural network is used to generate the affordance map of stacking objects,and then the grasp point with the highest probability of success is selected according to the evaluation of the affordance map.The experiment shows that MFCN(RESnet-101)is adopted to generate the affordance map,the positioning accuracy can reach 95.1%,and the detection speed can reach 10.2FPS.At the same time,it is shown that the method is independent of the recognition of soft packages,and can also locate the grasping points for the unseen soft packages.Then a soft packages recognition method based on YOLOv3 is proposed and tested in different fetching scenarios.The results show that,when the samples are sufficient,YOLOv3 algorithm has a very high classification accuracy(up to 96.2%)in relatively structured scenes,while the classification accuracy is greatly affected in complex scenes.At the same time,it proves the effectiveness of the "grasp first,look later " strategy.In addition,few shot learning method based on prototype network is proposed to meet the needs of insufficient samples and novel identification,which can realize the classification accuracy of 76.56% under 5way-5shot and is expected to adapt to the scenario of soft packages category expansion.Finally,we build an experiment system of robotic sorting for soft packages.The system consists of two Real Sense 435 D and a NACHI MZ04 robots.The sorting experiment shows that the proposed grasping positioning and soft packages recognition methods are effective,and the sorting accuracy reaches 95.4%.The two-stage strategy of " grasp first,look later " can improve the sorting success rate by 6.9% compared with the existing strategy of simultaneous identification and positioning.