A Study On Grasping Method For Mobile Manipulator Based On Deep Learning And Depth Camera

At present,automation technology is rapidly developing in the direction of intelligence,both in daily life and in industry.The development of intelligence is inseparable from intelligent algorithms,and it is also inseparable from its carrier,the robot.A superior visual perception system is one of the most convictive representations of intelligent robots.In recent years,with the improvement of computer property,big data and machine learning technology have been unprecedentedly developed,and a method from machine learning,named deep learning,has opened a door for the field of traditional machine vision.Although the object detection algorithm for deep learning has become increasingly mature,the development of applying this theoretical algorithm to actual robots has been relatively slow.This paper utilizes the mobile robot platform as the carrier to improve the Faster R-CNN object detection algorithm in deep learning,and establishes a new object detection network model.In addition,with the depth image information provided by the depth camera,the grasping point is found.Finally,the task of grasping the specified object is completed on the actual mobile robot.The main contents of this paper are:theoretical research on object detection algorithm in deep learning,the influence of ground truth in data set on training object detection network,designing Dataset-adaptive Algorithm for Choosing Anchor Box(DACAB),the construction and training of improved Faster R-CNN,and the location of grasping point and pose estimation based on the depth camera.Firstly,this paper construct a image datasets for 10 categories of daily commodities:toothbrush,toothpaste,chips,biscuit,milk,cola,tissue,shampoo,book and juice.This datasets include 4,000 images in total,400 in each category,in which each type of datasets is divided into training set and verification set at the ratio of 6:4 respectively.In the data set,various commodities are placed in different scenes with different pose and angles,which has been proved by experiments that a more robust object detection network model can be trained by such datasets.Secondly,based on the current mainstream object detection methods(YOLO,SSD,Faster R-CNN)in the field of deep learning,three object detection networks are established respectively,and they are trained on the daily commodity datasets,and the experimental exploration is designed to study the difference in performance among them.Finally,according to the datasets of this paper and the characteristics of Faster R-CNN object detection algorithm,the part of region proposal network is improved,and the DACAB is designed,thus forming the algorithm of improved Faster R-CNN object detection.Model A,B,C,D,E,F,G and H are trained for contrast test.Experiments show that the algorithm of improved Faster R-CNN object detection significantly reduces the time overhead of the original Faster R-CNN object detection without significantly reducing the accuracy of the detection,so that it can achieve the goal of real-time detection.Then,based on the depth camera's feature,providing useful depth image,and the predicted box of improved Faster R-CNN algorithm,combined with depth map and image processing method,the geometric center of the object is located.For the depth camera,object and mobile robot,the conversion relationship among their coordinate system is established,and the position of the grasping point on the object relative to the camera is obtained,which is used to complete the subsequent grasping experiments.Finally,a mobile manipulator system is built to verify the feasibility of the depth learning and depth camera based grasping method proposed in this paper,and we have carried out grasping experiments.The experiments is designed to study the influence of distance between object and robot manipulator on grasping success rate.So we did the grasping experiment repeatedly for 10 class at the distance between 10cm and 50cm 50 times(500 in total).The experimental results show that the proposed grasping method can be successfully applied to the mobile manipulator,and when distance equals 30cm,with the help of model F,the mobile manipulator successfully finished 431 grasps,it means success rate is 86.2%.