Application Research Of The Manipulator Based On 3D Vision And Deep Learning

In 2015,“Made in China 2025” was promulgated by the State Council.It was clearly stated in the document that robotics will be an important driving force in the field of high-tech in the future.As a significant branch of robots,manipulator has been widely applied in industries because of its great flexibility.In recent years,the rapid rise of e-commerce in China has led to the explosive growth of the parcel sorting business in logistics industry,and the traditional manual sorting mode is gradually being phased out.The manipulators,especially the intelligent manipulators with artificial intelligence technology,will play a great role in this field.In this thesis,the package was taken as the research target.The RGB-D camera,deep learning technology and an improved RRT path planning algorithm were applied to improve the traditional manipulator,and then realize the automatic detection and capture of the objects.The main research includes:1)Aiming at the actual package sorting process in logistics industry,the overall architecture of the intelligent manipulator system was designed.2)The depth camera was calibrated by Zhang's calibration algorithm,and the pixel registration between infrared and visual image was realized.3)Sample images were collected for YOLOv3 model training and testing.Parcel's bounding box calculated by YOLOv3 and the depth information captured from depth camera were combined to get the three dimensions and volume of the object.After that,the three-dimensional size and depth information of the object were used to calculate the position of the object's central point in the camera coordinate,which was applied to get the grab position.4)Move It simulation tool and AR label were used to carry out hand-eye calibration and realize space mapping from camera coordinates to manipulator base coordinates.Aiming at the shortcomings of the traditional RRT algorithms,an improved BRRT* path planning algorithm was applied to manipulator's grabbing and obstacle-avoidance research,and finally realized the automatic grabbing of parcels.In order to prove the effectiveness of the proposed intelligent manipulator system,experiments were carried out on each of its module.Experimental results show that its overall visual detection accuracy is high.For the five font facing types of packages,the recognition and classification accuracy rate is above 90%,and the average recognition accuracy rate is above 96%,which met the actual parcels sorting requirements.When measuring the objects' 3D information,the average error is less than 3%,and the accuracy is better than some existing structured light based methods.As for the motion control part,our path planning algorithm can effectively reduce the calculation time and optimize the final path quality when compared with traditional algorithms,with an average grasping success rate of more than 98%.Therefore,the intelligent manipulator system proposed can meet the actual automatic parcel sorting needs of the logistics industry.