Research On Viewpoint Estimation Based On Deep Convolutional Neural Networks

As robots are applied in increasingly complex scenes,the intelligence requirements for robots are increasing.Vision-based robotic systems should have the ability to actively perceive the complex unstructured environment.As a key part of active vision,the viewpoint estimation greatly affects the performance of following observation and operation of the robot.Based on the deep convolutional neural network(CNN),this paper has done a series of researches on viewpoint estimation.1.In order to prepare a large amount of training images for CNN in a simpler way,in this paper,CAD models and 3D rendering techniques are utilized to automatically generate a large amount of images from different viewpoints.Image processing techniques are then used to process the rendered images to make it more realistic.The CNN model has been proved to be feasible in the viewpoint estimation problem by training on these generated images.2.Compared with the normal image classification problem,there is a clear spatial relationship between the viewpoint classes.To make the CNN model more suitable for the viewpoint estimation problem,in this paper,metric learning is introduced,and a viewpoint-based triplet loss function is proposed to help the CNN model consider the spatial relationship between viewpoint classes to learn a more suitable feature space.The model is optimized by using spatial distance of viewpoint as supervision information,making it more suitable for viewpoint estimation problem.The proposed method is proved to be effective through experiments and visualization.3.To better address the domain discrepancy between synthetic images and real images and make the CNN model trained on synthetic images can be directly applied in real environment,in this paper,transfer learning is introduced,and a two-channel model combining the CNN algorithm and the unsupervised domain adaptation algorithm is proposed.The maximum mean discrepancy is used to quantify the difference between the real and synthetic sample distributions,and the viewpoint estimation task and the transfer task are completed in the model training process by optimizing the maximum mean discrepancy and the cross entropy loss.In addition,a eye-in-hand robot system is established to acquire a complete real viewpoint image data-set.The performance of experiments conducted under different conditions demonstrated that the proposed method can achieve promising adaptation performance under different conditions and the proposed method show its superiority over traditional image processing techniques.