Research On Limited Data-driven Light Field Salient Object Detection

Salient object detection(SOD)aims to identify the most attractive objects or regions in a scene.Current SOD algorithms can be divided into three categories based on RGB,RGB-D and light field inputs.Compared with the RGB and RGB-D data,light field data contains abundant scene information,which can satisfy the information demands for difficult scenes.In recent years,the development of deep convolutional neural networks has replaced traditional algorithms based on hand-crafted features and significantly improved the performance of light field salient object detection.However,in practice,the high cost of light field data acquisition,extremely complex light field multi-cue information processing and the time-consuming and laborious pixel-level annotation for SOD lead to the scaricity of current light field SOD datasets,making it difficult to provide sufficient data support for deep models.In response to the above problem,the thesis explores the limited data-driven light field salient object detection method from the perspective of efficiently utilizing of light filed information and augmenting light filed data.The main research work and innovation are as follows:(1)To address the challenge of how to efficiently utilize limited light field data,this thesis proposes a patch-aware network to explore light field data.This method mainly includes two modules: multi-source learning Module(MSLM)and sharpness recognition module(SRM).Among them,the MSLM takes the contribution of different regions of each focal slice to the prediction into account,and generates a region-wise attention weights under three guidances based on saliency,boundary and position for highlighting the salient regions in different focal slices.The attention weighs integrate the feature of focal stack.The SRM takes the influence of multi-focusness on saliency into account.It optimizes and updates the attention weights by identifying the focusness of each regions in focal slices.The salient regions can be highlighted and the non-salient regions can be suppress with the updated attention weights.Compared with the existing light field methods,the proposed method explores the light field data in a region-wise way,takes the contributions of different regions to the final prediction into account and utilizes light field information more effective.(2)To address the challenge of how to utilize limited light field data and saliency labels to augment the light field data,this thesis proposes a light field SOD method based on data augmentation.This method mainly includes two modules: the geometric augmentation module(GAM)and the focusness compensation module(FCM).Among them,the GAM recombines the salient objects and the background image to enhance the light field data by learning the geometric relationship of the scene through adversarial training.The FCM transfer the multi-focusness information to the style transfer network through adversarial training to optimize the multi-focusness details of the composited focal stack.In addition,the thesis proposes an uncertain learning strategy(UL)for the joint training of composited samples and real samples.The UL treat the data of varying quality unequally,which can get rid of the impact of low-quality data on network training.Compared with existing data augumenation methods,the proposed method can significantly improve the generalization and the detection performance of the baseline network.