Research On Light Field Saliency Detection Based On Convolutional Neural Networks

Saliency detection has always been an important research direction in the field of computer vision.Accurate and effective saliency detection can provide reliable prior information for visual tracking,object detection and recognition,redirection,image compression,etc.The saliency detection based on RGB and RGB-D images is easily affected by factors such as lighting and occlusion,and has low detection accuracy in challenging scenes.With the development of light field imaging technology,light field data has opened up a new way for saliency detection.The rich visual information in light field images,including color,depth,focus,position and direction of light,can help algorithms accurately locate salient objects in complex scenes.Therefore,the use of light field data for significance detection has attracted the attention of a large number of domestic and foreign scholars.However,existing light field saliency detection algorithms do not fully consider the correlation and complementarity between multi-modal light field data,resulting in unsatisfactory multimodal fusion performance and the occurrence of false positives and missed detections in complex scenes.In response to the above issues,this thesis utilizes the powerful feature extraction ability of convolutional neural networks to explore and utilize the rich visual information in micro-lens image arrays and focal stack to improve the detection accuracy of salient objects in complex scenes.The main work of this thesis is as follows:(1)A light field saliency detection algorithm based on micro-lens image array and allfocus image fusion is proposed.This algorithm first refines the micro-lens features to focus more on the information of angle changes and enhance their differences from all-focused features.Then,a light field fusion module is used to adaptively fuse the micro-lens features and all-focused features.By learning angle features from micro-lens image arrays,it helps distinguish foreground and background with similar colors and textures and improves the detection performance of the algorithm in complex scenes.(2)A light field saliency detection algorithm based on focal stack and all-focus image fusion is proposed.The algorithm first uses a dual-stream network architecture to extract features from the focal stack and all-focus images.Then,under the guidance of all-focused features,the contextual information within the focal slice group is propagated.Next,integrate the focal stack features by exploring the weight of each focal slice to emphasize useful light field features and suppress unnecessary light field features.Finally,the Conv GRU module is used to fuse the focal stack features and fully focused features.(3)This thesis conducts a large number of experiments to verify the effectiveness of the proposed algorithm.The proposed algorithm is compared with current advanced RGB,RGBD,and light field saliency detection algorithms,and ablation experiments were conducted.The experimental results show that the proposed light field saliency algorithm has strong robustness in complex scenes and has achieved excellent performance on multiple public datasets.