Scene Perception Method Based On Artificial Compound Eye

As a new type of imaging system,the artificial compound eyes(ACE),have a lot of features,such as wide field-of-view(FOV),small size and the ability of fast moving target detection.ACEs have the potential to be applied in machine vision community especially in the scene perception area based on tiny imaging system.In recent years,renewed attention has been paid to the artificial compound eyes,but most of the researches are focused on the design of ACE system.In scene perception area,because of the small aperture of ACE,traditional methods can not yield satisfied results on some problems,such as,ACE image rectification,three-dimensional reconstruction,motion estimation,et al.Therefore,in order to promote the ACE research from conception to practical application,this thesis based on a specific ACE-Electronic Cluster Eye(eCley),focused on the problems of ACE in system imaging,depth-based three-dimensional reconstruction under the static scene,optical flow estimation under the dynamic scene,from the way of algorithm model,algorithm optimization and experimental analysis to research and propose the scene perception method that suitable for artificial compound eye.Firstly,this thesis investigates the characteristics and imging distortion problems of eCley,and proposes an eCley image distortion correction method that considering the affects of the offset angles and unparallel incidents.Based on the parameters of eCley,this method estimates the undistortion image of each channel,then the distortion parameters are estimated based on the undistortion image and captured image,finally the inherent offset between adjacent channel images are estimated.The reconstructed image proofs that the proposed method is simple and efficient.Secondly,aim at the problem of three-dimensional image reconstruction of artificial compound eye under the static scene,this thesis proposes three 3D image reconstruction methods that considering the situation of disparity inconsistency caused by image distortion,textureless area of image,slanted surface,respectively.In order to avoid the inconsistent of disparity caused by image distortion,based on the parameters of eCley,the relationships of spatial coordinates of the pixels are deduced from the geometry aspect.The images of all the channels are then projected to space,and the matching costs are computed based on the spatial coordinates of pixels.The method is able to estimate the depth information and reconstruct the three-dimensional image in the mean time.The experimental results verify that this method can obtain satisfied three-dimensional image reconstruction result.In order to decrease the affect of image distortion and textureless area in depth estimation and improve the accuracy of three-dimensional image reconstruction.This thesis,based on the rectified image,proposes an edge-based depth matching and propagation method(EDMP)to accurately estimate the depth information of aperture images.The full three-dimensional reconstructed image is obtained based on the aperture images and their depth information.In EDMP,the edge information is used to obtain sparse depth map,and then the sparse depth maps are propagated based on fronto-parallel assumption to obtain dense depth maps.Since EDMP uses the rectified image to estimate the depth,the depth result is much better than G3D in some aspects in spite of the fronto-parallel assumption,and the reconstructed three-dimensonal images are better than G3D.In order to avoid the depth error in slanted surface caused by fronto-parallel assumption,this thesis uses the principle of the global stereo matching method,and proposes a cross image belief propagation based three-dimensional reconstruction method(CIBP).In CIBP,the aperture image is considered as a sub-image of a potential focused image,and a multi-aperture images optimization method that combines the corresponding area of adjacent images is designed to obtain consistent and improved depth result.Experimental results show that CIBP can obtain satisfied depth result at slanted surface and improve the results of three-dimensional reconstruction under slanted surface.Lastly,aim at the inaccurate optical flow at the border of aperture image in dynamic scene.Based on the idea of the intensity consistency of the corresponding area in CIBP,this thesis proposes a multi-aperture variational optical flow estimation method(MaVOF)to restrain the optical flow field of the corresponding area in adjacent images.The experimental results on Middlebury and Sintel datasets and images captured by eCley show that the MaVOF is able to suppress the inaccuracy caused by small aperture and object located in image border.