RGB-D Image Processing Method And Its Grasping Application Based On Fuzzy Attention Mechanism

With the rapid development of artificial intelligence technology,intelligent robotics has made continuous breakthroughs.The research on automatic recognition and grasp of robot is a typical problem in the field of intelligent robot,and has been the focus of research by scholars at home and abroad.The core technology of intelligent robot lies in the intelligent perception of environment and its coupling control with robot behavior.In this thesis,an improved convolutional neural network model is proposed and used to design a deep network based on RGB-D images.The object classification,intelligent grasp detection and grasping experiment verification are realized on the platform of Baxter robot.The main contributions of this thesis are as follows:(1)A new convolutional neural network structure,named multi-branch fuzzy architecture network(MBFAN),was proposed.MBFAN is a deep learning network structure designed based on T-S fuzzy model structure.The membership degree of parallel branches is obtained by using the attention mechanism network,and the characteristics of each branch are combined with the corresponding membership degree as the output of MBFAN network.In this part of the experiment,MBFAN was respectively combined with a simple CNN network and a typical VGG16 model to test on the CIFAR-10 classification dataset.Experimental results show that the identification accuracy of the network combined with MBFAN is improved compared with the original network on the basis of only adding fewer parameters.MBFAN uses the attention mechanism to increase the weight of important information branches,strengthen important features,and suppress unimportant features.The parallel branching design method in this thesis can effectively solve the problem of hyperparameter selection of convolutional neural networks.MBFAN is a feature extraction structure,which can be used in combination with most classical convolutional neural networks.(2)For RGB-D images,this thesis proposes two multimodal grasp detection models based on MBFAN: grasp detection model based on RGD and RGB-D,and verifies them on Cornell grasp dataset.RGD grasp detection model uses deep channel information to replace blue channel information and uses MBFAN combined with a network similar to Alex Net to grasp detection.The RGD grasp detection model retains the three-channel input feature,and the classic three-channel model can still be used,but the lack of blue channel will have an impact on the success rate of grasp.Therefore,a 4-channel RGB-D grasp detection model was designed on the basis of the RGD grasp detection model.In the RGB-D grasp detection model,RGB and D images are input into two convolution layers respectively,and then the output is concatenated in the full connection layer.This method avoids the problem of blue channel information loss and has a high success rate,but the number of network parameters is increased.(3)Based on the Baxter robot and the Kinect v2 depth camera,this thesis built a robot visual recognition and grasping system,and verified the method proposed in this thesis.In the experiment,Zhang Zhengyou calibration method and "eye-to-hand" calibration method were used to calibrate the camera and robot system,and Cornell grasp dataset was used to complete the training of RGB-D grasp detection model.The above experiments achieved 80% grasp accuracy,86% classification accuracy and76% comprehensive accuracy on the robot platform.Finally,the comparative experiment and theory are analyzed and prospected.