Research On Robot Stable Grasp Method Based On Visual And Tactile Fusion

Robot grasping has always been a challenging problem in the field of robotics research,especially in unstructured environments where unknown objects are scattered.Taking room arrangement task as an example,many challenges exists,including objects diversity,environment complexity and mutual interference between objects and environment.This paper mainly studies robotic stable grasping method for unknown multiple objects,focusing on the problem of unknown multi-object grasping,grasp quality evaluation and efficient stable grasping,including the following three aspects:(1)To solve the problem of predicting grasping parameters of multiple novel objects,a visual grasping method based on Mask RCNN and image principle axis computing is proposed.Firstly,the grasping planning problem in robot space is transformed to an object detection task in image space,and a complete grasping configuration of robot is established to characterize the grasping parameters.In order to realize the prediction of grasping parameters of multi-object environment,we use Mask RCNN for object detection and segmentation,and grasping parameters are calculated through principle axis computing of images.In the robot grasping challenge held at ICRA2018,this method achieves the successful rate of 100% for desktop objects grasping task.(2)To solve the problem that vision cannot be used to evaluate grasp quality,this paper proposes a quality evaluation method based on convolution operation of tactile image.The dot-matrix tactile sensor is installed at the end of the mechanical gripper to collect tactile data during physical contact operation and convert the data into tactile images.The convolution kernels with mathematical meaning are designed to make convolutions with tactile image,and the tactile quality can be acquired to evaluate grasp quality.Compared with Hogan metric,this method can be used universally,and it has better performance in object slippage detection to distinguish sliding state from stable state.(3)To solve the efficiency problem of robotic stable grasping,this paper proposes a stable grasping method based on visual and tactile fusion.The tactile prior knowledge learning model is used to establish the mapping between visual and tactile images,and then the model is integrated into the visual grasping framework to endow robot with tactile prior ability to realize efficient and stable grasping,rather than regrasping for many times.Compared with visual grasping,the stability of this method increases by55% while ensuring efficient grasping,and it shows good generalization ability.