| With the continuous development of artificial intelligence and deep learning technology,the autonomous grasp of robots plays an irreplaceable role in the fields of smart factories and intelligent homes.Current end-to-end grasp detection models suffer from repeatitive detection and poor robustness for multi-object.Therefore,this thesis proposes a method for serial processing of object recognition and grasp detection based on the fusion of RGB images and Depth images.This method supports the training of grasping detection models in single-object scenes,and can be directly applied to scenes with multiple objects.The method can increase the generalization of grasp detection model and relieve the workload of manually labeling data.The main contents are as follows:Firstly,the detectors based on RGB images are prone to missed detection and false detection for densely occluded scenes.Therefore,a Sub-stage Path Aggregation module(SPA)and the SPA-YOLO-Fusion object detection model based on RGB-D feature level fusion are proposed.Based on the YOLOv3-tiny,the SPA-YOLO-Fusion utilizes two different feature extraction networks to extract features from the RGB image and the Depth image respectively.Two features are processed by feature concatenation and the SPA module is used to fuse the concatenated features.The Line MOD dataset has been re-labeled and the YODO＿Det dataset is created from the images collected from the actual scenes.The experimental results show that the accuracy of the SPA-YOLOFusion model on the two datasets is 91% and 95% respectively,and the illumination robustness and generalization of the feature fusion module are also better than the lightweight models of YOLOv3 and YOLOv4.Secondly,based on RGB-D image pixel-level fusion,GR-Conv Net grasp detection model is improved from three aspects: loss function,image preprocessing algorithm and post-processing algorithm.This thesis proposes the MS-L1(Mask Smooth L1)loss function for training,which assigns greater weight to the object area images outputted by the SPA-YOLO-Fusion.In order to improve the grasp detection accuracy of densely occluded objects,this thesis proposes an image preprocessing algorithm that fills background with white and a post-processing algorithm that filters image pixels outside the objects area.Images have been collected from the laboratory to get YODO＿Grasp dataset is created,which is characterized by the fact that the training set is single-object images and the test set is multi-object images.Results show that the accuracy of the improved GR-Conv Net and GG-CNN is 85% and 73%,improved by 23% and 17%respectively.On the platform equipped with the RealSense D435i and the Jaco2 robotic arm,object detection and grasp detection have been carried out on both single-object scenes and multi-object scenes densely with placed six types of objects,and then 20 grasp experiments alse be carried out on the effective grasp boxes.The results show that the object detection m AP is 97%,the grasp detection accuracy is 97%,and the grasp success rate is 93% in the single-object scene.In the multi-object scenes,the grasp detection accuracy is 75%,the grasp success rate is 90%,outperforming other models.Finally,a software is developed to realize the functions of object detection and grasp detection based on RGB-D fusion.This software improves the ability of visualization and the operability of the grasp detection task.The thesis contains 61 figures,7 tables and 92 references. |
