Object Recognition And 6D Pose Estimation Method Based On 3D Multi-View Data Generation

Recently artificial intelligence technology is booming,many intelligent robots have come into our life,such as medical robots,industrial robots,home service robots and so on.For these robots,object recognition and 6D pose estimation are very impor-tant abilities.It is an important prerequisite for robots to perform lots of tasks such as grasping and sorting.Recently,deep learning has achieved great success in all major fields,especially in the field of computer vision.Therefore,how to apply deep learn-ing to object recognition and pose estimation in robots has attracted a lot of research interest.On the one hand,this paper mainly studies the generation method of multi-view simulation data and its application in the field of object recognition On the other hand,A multimodal feature fusion pose estimation method which is driven by semantic seg-mentation is proposed.The generation of multi-view simulation data is used to train the object recognition network and the 6D pose estimation network,which is especially important in the case of lack of training data.Semantic segmentation network extracts RGB image features and point cloud features from the origin images,the multimodal feature fusion network fuses these two features to predict 6D pose of object,we can get more accurate result after optimizing further.The details are as follows:(1)Generation of multi-view simulation data and its application in object recogni-tion.In this paper,a simple and easy method is used to build 3D model of the object.Specifically,the target object is placed on the plane of the turntable,Kinect sensor col-lects the colored point cloud data when the turntable is rotating.After the rotation of the turntable is completed,we match these collected point clouds,and finally a complete 3D colored point cloud model of the target object is obtained.Secondly,we take plenty of photos of the target obj ect model from different perspectives,at the same time the class category and pose annotation information of the object can be obtained.Finally,this paper uses these simulation data to train the recognition network to verify whether the method of data generation is effective.Experiments show that this method can achieve higher accuracy and has good generalization performance for real pictures.(2)Pose estimation method for industrial logistics sorting scenario.In this paper,a pose estimation method based on principal component analysis is proposed.It is simple and fast,and can be applied to estimating 6D pose of objects with regular shapes.For the irregular shape objects,a multimodal feature fusion method driven by semantic segmentation is introduced.The RGB image and point cloud of the object are obtained by the semantic segmentation network to avoid the inference of complex environment,then the 6D pose of object is obtained by using the neural network to extact and fuse the features of the image and point cloud,finally we can further optimize the preliminary prediction of the model to get more accurate result This paper trains and tests the nerual network model on the LINEMOD dataset.Compared with the best result so far,the method has faster speed and higher accuracy in the experiments.