Recognition And Grasp Laser-Pointed Object Based On RGB-D Camera

As the population aging becomes more severe,the demand for a wheerchair mounted robotic arm(WMRA)is increasing day by day.In order to satisfy the pursuit of better qualit y of life for the disabled and the elderly,and relieve nursing pressure of paramedics,therefore a WMRA is developed to help users eat,and open doors,etc.How to achieve frie ndly human-robot interaction and successful grasp based on interactive information is the priority for robots to solve when they perform grasping tasks.Therefore,a laser-guide autonomous grasping method is proposed in this paper.Moreover,the robot uses You O nly Look O nce v3(YO LO v3)algorithm to detect laser point,the depth point cloud to extract and locate the object,and then completes the grasping according to the teaching information of the object.In addit ion,the key technologies mentioned above are integrated into the Robot Operating System(ROS),and laser-pointed grasp based on RGB-D information is done at last.The specific research contents are as follows:(1)The precise acquisit ion of laser point in the environment.Firstly,laser point detection was realized by improved target matching and background subtraction method,but the method of labeled wafer on the object in the detection process restricted its use.Aiming at the problems of traditional detection algorithm,the Convolut ional Neural Network was used to realize laser point detection.YO LO v3 algorithm was used to detect laser point,image processing was carried out at the front of YO LO v3 network,and prediction frame of the pointed ob ject was roughened and discolored in visualization at the end of YO LO v3 network,which improved the users' interactive feeling,increased the success rate of laser point detection.(2)The mathod to recognize and grasp laser-pointed object was researched.Based on the RGB-D image information,filtering and segmentation algorithm us ing object point cloud information was studied,and three-dimensional centroid coordinates of objects in the environment were extracted.The transformation equation between coordinates of objects' point cloud and laser point was built to determine the three-dimensional position of the object attached to laser point,and the recognit ion of objects was obtained based on C onvolutional N eural Network.What's more,the pose of laser-pointed object was achieved by multi-template matching algorithm,and trajectory planning was used to grasp pointed object at last.(3)Laser-point interactive grasping system and coordinates transformation relationship to achieve accurate three-dimensional pose matching of laser-pointed object and template objects.The relationships among coordinate systems in the autonomous grasping system were described,and coordinate systems were calibrated.By matching RGB color image with depth image,the relationship equation between two-dimensional coordinates of laser point and three-dimensional centroid coordinates of point cloud object was deduced,and the precise matching between laser-pointed object and three-dimensional point cloud object and the understanding of point cloud scene were realized.(4)The WMRA robot system was built and the experiment of laser-point interaction was studied.The software integration and hardware construction of the system were completed based on the robot operating system(ROS).O n the laser-point grasping platform,the experiments of laser-pointed object detection and intelligent grasping experiment of laser-pointed object were carried out.Moreover,the users' utilize experience of the intelligent interactive grasping system were analyzed.