Semantic Integrated Pose Estimation In Unstructured Environment

With the development of science and technology development strategies such as “Industry 4.0” and “Made in China 2025”,Chinese industry has placed an urgent need for the intelligent manufacturing upgrade of traditional industries.Application scenarios such as automated logistics sorting and industrial automation sorting,an efficient and reliable object pose estimation techniques are of great significance for robot applications.In this thesis,we study on pose estimation problem in unstructured environment.The specific content includes the following four aspects:(1)Aiming at the problem of high cost of dataset construction in practical deep learning applications,we propose an object semantic dataset automated construction method based on physical simulation.Each data sample includes color maps,depth maps,and point cloud data of the simulated scene.The category,bounding box,semantic segmentation,part semantic segmentation and pose annotation of the objects are extracted in the corresponding scenes.(2)Based on the object semantic dataset,a part semantic segmentation network is proposed,which can detect the object semantic segmentation and object part semantic segmentation in the image.This thesis improves the instance semantic segmentation network structure and realize the finegrained objects semantic segmentation by adding part semantic branch and designing corresponding loss function and training method.Experiments show that the network can obtain effective detection results on the simulation dataset.And we can obtain effective detection results when applying the network trained on simulation dataset to the real scene dataset.(3)Based on the fine-grained semantic object detection results,we propose a semantic integrated unstructured environment object pose estimation algorithm.This thesis improves the point pair feature based object pose estimation algorithm in multiple stages.At the same time,we design the semantic point pair feature based pose estimation algorithm.Combined with the fine-grained semantic detection results,a semantic integrated object detection and pose estimation framework is proposed.Experiments show that speed and accuracy are effectively improved compared with the traditional pose estimation algorithm.Besides,it can solve the situation that cannot be solved for traditional point cloud based pose estimation algorithm.(4)In order to verify performance of semantic integrated pose estimation algorithm in real unstructured scenes application,we design a robot intelligent picking system based on ROS.Through the ROS distributed framework,the modules,such as visual algorithm and robot control,are decoupled.Each algorithm module forms an independent node which enhances the scalability of the program.Experiments show that the robot hand-eye system and object pose estimation algorithm proposed in this thesis can successfully complete the intelligent picking and sorting.