| In recent years,the use of service robots in life is becoming more and more widespread.Desktop robotic grasping is one of the main research directions of service robots,and is widely used in life tasks such as caring for disabled and elderly people,meal preparation in restaurant and selling coffee.Therefore,the research on the daily goods grasping of the desktop service robotic arm has important application value.The service robotic arm works in a living environment.Compared with the industrial manipulator used in a structured environment,its grasping system requires better environmental perception and autonomous decision-making ability.In view of it,this thesis applies machine vision technology in robotic arm to research the grasping of daily goods including grasping,stowing and pouring water.The main research contents are as follows:Firstly,the Kinova Jaco2 which has six-axis lightweight bionic robotic arm with three-fingers clamps is selected as grasping execution device,after analyzing the requirements related to the grasping of daily goods and considering safety and grasping stability.The hardware platform for grasping experiments is buid by combining Kinova Jaco2 with Realsense D435 i,an economical RGB-D depth camera.The platform for algorithm testing and simulation verification is developed based on ROS.The experimental platform supports the development of algorithms based on the combination of virtual and real.The algorithms can be verified in the simulation platform,and then tested on the actual platform.Grasping detection algorithms are mainly developed by using Opencv and Pytorch.In order to obtain a stable and wide grasping field of view,the positional relationship between the camera and the robotic arm is eye-to-hand.The Epnp calibration method is used to complete hand-eye calibration experiment.Secondly,through the object detection methods based on image matching,the grasping and stowing of a small number of specific daily goods are studied.Image matching technology recognizes specific objects by designing features.It has strong interpretability and convenient parameter adjustment.It has very high efficiency and practicality for some specific objects and scenes.In the research of object detection,template matching is used to detect objects with fixed or small changes in pose.For most daily goods with textures,the combination of image sharpening,region of interest extraction,and Knn-matching filtering error point pairs is introduced to improve detection performance of existing SURF matching algorithm.For non-textured objects,the Canny operator and Hu moment feature are used for matching detection because SURF algorithm detection is not robust.The experimental results indicate the effectiveness of the three algorithms for detecting objects with relevant characteristics.In the research of pose estimation,firstly,the depth map is aligned and the hole is repaired.Then the depth value is used for the three-dimensional positioning of the target.Finally,the plane rotation angle is used to calculate the posture of the object on the desktop,which simplifys the posture estimation algorithm.In the research of grasping detection,according to the characteristics of the three-finger manipulator,the vertical grasping posture is adopted to grasp the objects,and the grasping detection configuration is designed based on grasping points and finger opening degree.The experimental results of multi-target grasping indicate the effectiveness of target detection,pose estimation and capture detection methods.Thirdly,in view of that it is difficult for traditional image matching detection technology to handle with the grasping problem of multiple types and large numbers of daily goods,deep learning technology is used to perform object detection.YOLOv5 and K-means segmentation are proposed to achieve better detection results and more refined grasping point.In order to train the deep learning-based detection model,based on the Cornell grasp data,the data is constructed through web crawlers,selfie images,etc.Label Img is used for labeling,and a data set of 11 daily goods was produced.The object detection performance of Faster-RCNN,SSD,Center Net and YOLOv5 models is compared on this dataset.The results show that YOLOv5 not only has better recognition performance,but also has robustness and generalization.It also works well in grasping scenes that are different from the training dataset environment.When grasping,relative to the vertical grasping posture of a small number of specific items,a horizontal grasping posture is added to improve the grasping ability under the variety of items.The experimental results in the complex grasping scene show that the robotic arm grasping based on YOLO object detection can complete grasping and stowing applications of multiple quantities and classes of daily goods.Fourthly,the generalization of the robotic-arm grasping scheme used in this research is verified by a water pouring task.The YOLOv5 model is applied to detect the liquid in the water cup and the water cup at the same time,and the liquid level height is determined by the spatial relationship between the water cup and the liquid.The proposed model has good generalization and can detect water cups and liquids that do not appear in in the training and validation sets.In the process of pouring water,a more accurate detection of the pouring position is proposed by adopting a horizontal camera configuration,and the speed of the wrist is controlled to improve the success rate of pouring water.Success rate of 77.5% is obtained in the pouring experiment of four liquids: mineral water,milk,juice,and cola.The experimental results show that the robotic arm grasping program has task expansion capabilities.Finally,the upper computer grasping software is designed based on the above research content.The software integrates the detection and grasping algorithms of this research,and develops a human-computer interaction interface based on QT.The paper has 98 pictures,17 tables and 95 references. |
