Robotic Hand-Eye Coordinative Grasping Technique Based On Visual Attention

In recent years,artificial intelligence technologies,like deep learning,have been undergoing a wave of booming development and made fruitful achievements in fields like image classification,speech recognition,and natural language processing.However,robots,which play an important role in mechanical manufacturing and other fields,still rely on manual teaching or precise programming based on camera calibration in tasks like grasping,handling and palletizing,and the manipulating objects are often limited to objects with the same appearance,which is far from meeting the actual needs.Therefore,it is imperative to improve the intelligence level of robotic manipulation by introducing deep learning algorithms.This paper proposes and verifies a series of algorithm improvements for robotic hand-eye coordinative grasping technique.On the basis of researches on robotic hand-eye coordinative grasping system,this paper proposes to use recurrent neural network based on visual attention mechanism as grasp success chance prediction model,which is the kernel module of the system.Taking advantages of the characteristic that the network expresses the positions of image features explicitly,we concatenate the position of image features in 2D image coordinate system and the robot moving vector in 3D base coordinate system,and then input them into the network.This helps the model to learn mapping relationship between two coordinate systems,endues the model with interpretability,and increases the accuracy with less calculation.To solve the problem in model training caused by quantity imbalance between positive and negative samples,this paper analyses the characteristic of the value distribution in crossentropy loss function,and proposes to replace sigmoid classifier with linear support vector machine as classifier in the grasp success chance prediction model,regarding the function margin as success chance.We give the loss function and gradient updating formula,and validate the improvement on model performance.In continuous operation space,it is difficult to figure out the robot moving vector that makes the grasp success chance prediction model outputs largest value.We employ the sampling based cross-entropy method(CEM)to help search the optimal moving vector,and give the formula derivation and implementation details for this task.Based on the function margin output by the grasp success prediction model,we design a visual servoing mechanism to decide whether to move,taking the difference between success chances of grasping at positions before and after moving along the optimal vector as criteria.Experiments prove that the visual servoing mechanism is capable of producing a robust and efficient grasping procedure.Based on the forementioned research,we developed the hardware and software of the robotic hand-eye coordinative grasping system,accomplished date collecting,model training,model evaluation and grasping experiment,validating the effectiveness of the algorithms.