Research Of Grasping Dataset Construction And Grasping Salience Prediction For Unknown Objects

Grabbing saliency graph is an important analysis tool to explore human grasping skills.It usually reflects people's grasping tendency between different parts of an object in daily life.For the same target object,the higher the significance of some part of it,the more people are accustomed to or more inclined to choose this part to complete grasping operation.Grasp saliency has many potential applications in the field of vision and robotics.For example,it can assist the robot to select the grasp point of the target object,so as to complete the grasping operation better and faster.In order to obtain the saliency graph,it is usually necessary to complete the experiment setting,operation and subsequent processing with the help of the data set.Therefore,the quality and content of the data set have a direct or indirect impact on the result of saliency graph capture to a large extent.For the robot grasp,a perfect grasp can be completed only by the cooperation of the precise vision system and the flexible manipulator arm.Although the current robot grasping technology has been relatively perfect,there are still many problems,such as the rigid robot arm due to the unsmooth grasping path planning,the weird grasping operation,and the inability to select a better grasping point to complete grasping when facing the unknown object.It is a continuous and very natural process for human hands to complete grasping operation,so it has become our goal to humanize the grasping operation of the robot.As the basis of grasp and manipulation,the movement trajectory of human hand in the process of approaching the target is very important,the existing grasping data sets only focus on the acquisition of contact points during grasping,ignoring the shape,pose and movement trajectory of the hand during grasping.In addition,the experimental environment and conditions are relatively single in the setting of data collection,which makes the data set lack of diversity.For grasping saliency graphs,previous scholars have obtained relatively complete grasping saliency training sets of models(such as recording grasping contact points by means of target tracking and thermal imager),and have been able to predict the saliency graphs of unknown objects through deep learning and other methods.However,due to the relatively single data set used,the prediction of unknown objects is also relatively limited.When the external conditions change(such as the objects at different heights or at different angles),the previous work can not make a good prediction.In view of the above problems,this paper builds a rich variety of grasping data set,and trains the neural network to predict the grasping salience of unknown objects based on this data set,which overcomes the defects of single data set and very limited prediction results in previous work.To be specific,considering the factors of robot oriented grasping path planning,grasping point selection and grasping operation anthropomorphization,volunteers were recruited and the grasping data set was made,which not only recorded the information of hand joints and contact points during grasping,but also recorded the track information of hand movement before grasping.For the same set of models,the grasping information at different heights and angles is recorded.After filtering and processing the collected data,the grasping salience of the model is calculated.Then,the grasping salience of the known model is used for deep learning by PointNet++.Finally,the prediction of the grasping salience of the unknown model can be realized in a more varied manner.