Research And Implementation Of Omnidirectional Scene Segmentation Based On Fully Convolutional Networks

With the continuous improvement of computer performance and the coming of big data era,deep learning has received more and more attention.It explores the abstract high-level features by a combination of low-level features so that it can describe the data more intrinsically.Therefore,deep learning has been widely used in various fields,such as computer vision,speech processing...Convolutional neural networks(CNN)is a kind of deep neural networks with the characteristics of sparse connection and shared weights.CNN can automati-cally extract features and mainly used in recognition of images,it can achieve the classification in image level.Fully convolutional networks(FCN)is the extension of CNN,it adds some new features,such as Fully Convolutional,Upsampling and Skip Architecture.FCN can predict the semantic labels of each pixel,it achieves the classification in pixel level.In this thesis,we respectively discussed the structure,principle,characteristics of CNN and FCN.In order to achieve omnidirectional scene segmentation,we introduced panoramic images in the thesis and then analyzed the types,features,applications,synthesis methods of panoramas.In order to further improve the result of omnidirectional scene segmentation,we designed a new network based on FCN.The network attained a better ability of handling the edges and details because of the combination of panorama and depth image.The main innovation points of this paper are as follows:First,we replaced the normal images with spherical panoramic images,these panoramas had a broader vision,which could completely cover the surrounding environment and provide more abundant information about space,location and others.It could help the network to extract more representative features.Second,we improved the structure of the neural network based on FCN.Parallel-input was used to deal with RGB images and depth images,on the upper layer,RGB image was processed to obtain integral space framework;on the lower level,we introduced the depth image as the constraint,took advantage of the obvious contour feature and simplicity of depth image,then extracted relatively fine external geometric features by convolution networks.We combined different feature maps of the same scene to improve the accuracy of scene segmentation.