Research On Occlusion Detection And Avoidance Methods Based On Pixel-level Feature

Whether it is passive vision or active vision,and no matter how many cameras or types of cameras are used in the vision system,in most vision research fields,such as object recognition,3D reconstruction,object tracking,motion estimation,visual observation,scene rendering,robotic grabbing,automatic assembly,spacecraft docking and other fields will involve occlusion.Once the occlusion phenomenon occurs in the vision system,it will affect the object recognition,reconstruction,tracking,observation and other operational effects,and even cause the related operational tasks to fail.The occlusion problem has become a bottleneck problem in the field of visual information processing.For different types of occlusion problems,this research starts from the relevant pixel-level feature of occlusion detection and avoidance,and aims to study the methods of occlusion detection and avoidance based on pixel-level feature.The specific research work is as follows:Firstly,the related concepts and technologies involved in occlusion detection and avoidance methods based on pixel-level feature are introduced.The key basic technologies and concepts required for the research on this subject,such as the random forest classifier model for pixel-level feature classification,graph cut theory to solve the occlusion detection energy function,K-Means clustering algorithm for multi-factor video pixel occlusion category classification,depth image,deep neural network infrastructure about extracting occlusion learning feature of neural network for depth image,pixel-level Gaussian curvature feature of depth images,and SIFT matching algorithm are described in detail,which lay the foundation for the subsequent chapters of occlusion detection and avoidance methods based on pixel-level feature.Secondly,to solve the problem that the existing occlusion region detection methods for color video can not make full use of all kinds of occlusion information in the video,starting from the research on the pixel-level occlusion region feature for the color video,an occlusion region detection method is proposed for color video by fusing multi-feature based on graph cut.Research on the pixel-level occlusion region feature for the color video,three new pixel-level occlusion region features named brightness patch match,maximal flow difference and flow residual are proposed based on the information of optical flow and brightness,and meanwhile their calculation methods are defined.The feature vector of each pixel is composed of the proposed features and is inputted into the random forest classifier to obtain the occlusion information about pixels and adjacent pixel pairs.An occlusion detection energy function,which transforms the occlusion detection problem to an optimization one,is constructed by synthesizing the above occlusion information.An undirected graph is constructed according to the energy function,and then the energy function is solved by graph cut theory to gain the final occlusion region detection result.The experimental results show that,the proposed method has higher accuracy and better real-time performance.Thirdly,to solve the problem that the the existing occlusion region detection methods cannot effectively detect the occlusion region for complex video scenes containing multiple factors such as visible region,shadow,noise,etc.,starting from the research on the multi-factor pixel-level occlusion region feature for the color video,a unsupervised occlusion region detection method for video based on multi-factor feature is proposed.The multi-factor occlusion determination principle is proposed and defined,and based on the multi-factor occlusion determination principle,research on the multi-factor pixel-level occlusion region feature for the color video,and using the information for optical flow,brightness and three-channel of RGB between two frames,two multi-factor pixel-level occlusion region features named brightness patch change ratio and three-color attenuation ratio are proposed and defined.The multi-factor pixel-level occlusion region features are normalized,and the cluster centroids for the pixels of the training sample by K-Means clustering are obtained.A feature weight calculation method is proposed and defined.Based on the calculated feature weights,an improved distance calculation formula is designed,and weighted K-Means clustering based on the improved distance calculation formula is realized to gain the final multi-factor occlusion region detection result.The experimental results show that,the proposed multi-factor video occlusion region detection method is effective,thus solving the occlusion region detection problem for complex video scenes containing multiple factors such as visible region,shadow,noise,etc..Fourthly,because the depth image contains rich distance and position information of the visual object,on the basis of the research of occlusion detection for color video,the occlusion boundary detection method for depth image is researched,and starting from the research on the pixel-level occlusion manual features and learning feature of neural network for depth image,an occlusion boundary detection method for depth image based on the fusion of manual features and learning feature of neural network is proposed.Based on the information of depth and spatial location,research on the pixel-level occlusion manual features for depth image,and two pixel-level occlusion manual features for depth image named depth gradient and angle dispersion are proposed and defined.Research on the pixel-level occlusion learning feature of neural network for depth image,and a suitable deep neural network framework is constructed to extract the occlusion related learning feature of neural network for depth image.On this basis,the extracted manual features and learning feature of neural network are weighted fused based on feature importance and weight distribution ideas.The weighted fusion features are classified by random forest classifier to gain the final boundary detection result for depth image.The experimental results show that,the proposed occlusion boundary detection method is effective.Finally,for the problem of unknown information acquisition when the moving object is occlued in the depth image video,an approach for dynamically avoiding occlusion fusing pixel-level feature based on the depth image sequence of moving visual object is proposed.Two adjacent depth images of a moving object are acquired and each pixel's 3D coordinates in two adjacent depth images are calculated by utilizing anti-projection transformation.On this basis,the best view model is constructed according to the occlusion information in the second depth image.Research on the pixel-level feature representation of surface shape invariance for the moving visual object in the depth image,and the Gaussian curvature feature matrix corresponding to each depth image is calculated by using the pixels' 3D coordinates.Based on the characteristic that the Gaussian curvature is the intrinsic invariant of a surface,the object motion estimation is implemented by matching two Gaussian curvature feature matrices and using the coordinates' changes of the matched 3D points.Combining the best view model and the motion estimation result,the optimization theory is adopted for planning the camera behavior to accomplish dynamic occlusion avoidance process.Experimental results demonstrate the proposed approach is feasible and effective.