Study On Scene Features Recognition And 3-D Reconstruction Of Large Field Of View For Night Vision

In the field of robotics,a lot of operations in harsh environments are usually carried out at night,which are difficult to complete manually without light.The field of view and the ability of scene recognition of robots determine the task execution capability of robots.In the night mode,infrared imaging system is usually used to obtain night scene images.Infrared images are monochromatic and with low signal to noise ratio,low contrast and lack of depth.Moreover the infrared image has narrow field of view and is difficult to meet the needs of wide view angle observation applications.Therefore,it is meaningful to study how to extract and identify scene feature information of infrared images,how to extend the observation field of infrared images and display the 3D scenery for robot vision at night.Based on the properties of infrared imaging,this thesis focuses on infrared scene feature recognition,large field view mosaic of infrared image and three-dimensional reconstruction.The main research content includes three parts: the first part is the infrared scene feature recognition algorithm based on the deep convolution-deconvolution neural network;the second part is the infrared image mosaic algorithm using point feature operators and the multi-infrared images mosaic algorithm;the third part is 3D reconstruction of large field of view of infrared images based on 3D reconstruction principle of monocular infrared images.The main contributions and innovations of the thesis are as follows:1.We propose a novel algorithm based on deep convolution-deconvolution neural network for infrared scene feature recognition.The architecture of the deep network consists of VGG16-based convolution and deconvolution layers.Based on the learning of the training samples,it can implicitly to learn effective infrared scene features from the training data,without explicit feature extraction.The trained model can identify nine kinds of targets,such as sky,architecture,trees and so on.We compare our proposed algorithms with the existing recognition algorithms.The comparison demonstrates that our proposed algorithm obtains higher pixel accuracy,class accuracy and mean IU,which has better recognition performance for the test infrared images.2.In the monocular multi-angle scanning scene acquisition mode,multi-infrared images mosaic is implemented in two steps.First,mosaic technique of two infrared images are studied and a novel algorithm based on the point feature operator is proposed for infrared image mosaic.The displacement of adjacent image is estimated using the phase correlation method,and the feature points in the coincidence region are extracted for image matching,which effectively shortens the matching time and improves the efficiency of mosaic.At the same time,multi-resolution weighted fusion algorithm is used to mosaic the image.Second,multi-infrared images mosaic is performed based on the above algorithm.In order to eliminate the accumulated errors caused by cascade transformation of multi-images mosaic,an LM-based optimization algorithm is proposed to continuously adjust the homographic matrix of the infrared image to the reference plane during the stitching process,which can improve the quality of mosaic.A 180 ° seamless large field of view infrared image is obtained after the above steps.3.In this thesis,we combine multi-infrared images mosaic algorithm and 3D reconstruction principle of monocular infrared image,which is based on panel parameter Markov field(PP-MRF),to reconstruct large field of view infrared image.The stereoscopic and large field of view image display is obtained which can reflect the spatial depth information of the infrared scene.This is very meaningful for the remote commander to understand the whole scenery and the positions of targets.