| With labor costs rising and demographic dividends declining,society’s demand for robots is becoming more and more urgent.There is a growing tendency to use machines to replace human work in fields such as industry,agriculture,and medicine.For humans,grasping action is an instinct,and humans can grasp a specific target stably and accurately even in a dynamically changing environment.But for robots,grasping tasks are still very challenging scientific problems because they involve the environment perception and modeling,motion planning,and executive control.This paper mainly focuses on robotic multi-task grasping for stacked scenes,combined with deep learning theory and technology,focusing on the research and realization of the detection of grasping region and the recognition of grasping order of the robot in the stacked scenes.This paper will study the following aspects:1.Aiming at the problem of difficulty in grasping detection due to excessive overlap and occlusion of objects in stacked scenes,a new method for robotic grasping detection based on Region Proposal Networks(RPN)is proposed.The research method first extracts the Region of Interest(ROI)of the grasping scene,and then the deep learning grasping detection is further implemented on the ROI feature map.The grasping detection idea proposed in this paper is object-based ROI feature learning,which is different from the traditional learning method based on the feature learning of the entire grasping scene image,so as to effectively eliminate the influence of the cluttered background and effectively improve the robustness grasping performance of the robot in the stacked scene.Validated on the Visual Manipulation Relationship Dataset(VMRD)and Cornell Grasp Dataset,the results show that the research method is comparable to the current state-of-the-art grasping detection algorithms in single-object scenes,while achieving better grasping detection performance compared to existing methods in densely stacked scenes.2.Aiming at the problem of difficult matching between the stacked multi-object grasping detection and its corresponding object category,a new multi-task deep network structure is proposed.By adding an object detector after the ROI pooling layers,the object detection and grasping detection are realized at the same time.The object detector and grasp detector take the ROI features as input.The network model can effectively determine the object category,and effectively match the corresponding relationship between the detection object and grasping position in the densely stacked scenes,so as to realize the robot’s intention to grasp objects of different categories in the stacked scenes.3.Aiming at the problem of grasping detection anchor box orientation,a rotation angle-based directional anchor box mechanism is designed,which sets multiple directed prior rectangles with different default rotation angles in each region of the input image as the final grasp detection benchmarks to improve the accuracy of grasp detection.Considering the limitations of the current Jaccard index,an effective directional anchor box matching strategy is proposed.The new anchor box strategy has lower computational complexity and can better adapt to the proposed method framework and give more accurate matching result.4.Aiming at the problem of low reasoning accuracy of robotic grasping for objects manipulation relationship in the stacked scenes,a manipulation relationship reasoning algorithm that integrates visual features and location information is proposed.By designing the Object Pairing Pooling Layer(OP~2L),the end-to-end training of object detection and manipulation relationship reasoning is realized,and the speed and performance of the grasping detection algorithm are improved.The algorithm traverses the output results of the object detector,calculates the union bounding box image position of the object pair,improves the current situation that only visual features are used as reasoning clues,and extracts the relative location features between objects as a supplement to the visual features,reducing the probability of misjudgment of manipulation relationship.Through comparative experiments,it is proved that this method can effectively improve the accuracy of identifying the manipulation relationship of the object pair,and enhance the model’s reasoning ability,so as to improve the accuracy of identifying the grasping order.5.Combining the above proposed grasping detection algorithm and manipulation relationship reasoning algorithm,a deep framework for robot perception,reasoning and intention-grasping in object densely stacked scenes is designed.The method first uses the scene image as input to realize object detection and grasping detection,and then uses the manipulation relationship reasoning algorithm to generate the manipulation relationship tree,and traverses the manipulation relationship tree to obtain the multi-task grasping order of the current stacked scenes.Then through the system calibration and coordinate transformation model,the obtained object pose information in the camera coordinate system is converted into the pose information in the base coordinate system of the manipulator,and the robot inverse kinematics model is used to realize the robot grasping motion planning.Finally,the feasibility of the proposed method is proved by the robotic grasping manipulation experiments in different scenes.The experimental results show that the method can accurately complete the grasping detection in the stacked scenes,and enable the robot to grasp the specified objects in the order of intention. |
