Stereoscopic Video Saliency Detection Based On Depth Information

It is the main trend that computer intelligence helps human analyze and process image data.As an important technology in computer vision,saliency detection enables computer to simulate human's visual attention mechanism,and quickly perceive and extract interested areas in scenes.With the development of depth camera,the acquisition of depth map can further suppress the background.Stereoscopic video saliency detection has been developing,but it is still facing great challenges.The difficulty mainly lies in the lack of a benchmark dataset,the manual annotation of ground truth,the diversity of information,the diversity of motion,the complexity of background and the accuracy,real-time and stability for the detection requirements.According to the above issues,this paper establishes a stereoscopic video saliency dataset,generates the ground truth by semi-supervised method,and devises two detection algorithms based on traditional and deep neural network.The main work of this paper is as follows:(1)Aiming at the lack of stereoscopic video saliency dataset,a depth camera is employed to capture the data set.And a calibration method and a hole filling algorithm are designed to get the depth map corresponding to the color map one to one.Aiming at time-consuming and laborious problem of the manual annotation of ground truth,a semi supervised network is employed to generate all the ground truth automatically.(2)A stereoscopic video saliency detection algorithm based on spatiotemporal correlation and depth confidence optimization is devised.In order to accurately detect salient objects in consecutive frames,the spatial correlation is enhanced by the interaction between neighbors,and the temporal correlation is enhanced by propagating motion information in sequential frames and reverse sequential frames.Then,the depth confidence optimization is proposed to fuse the spatial saliency,temporal saliency and depth with high accuracy.(3)A semi supervised algorithm based on two-stream network is designed for stereoscopic video saliency detection.In order to achieve the real-time requirement,we use the two-stream network to extract the spatiotemporal information and depth information respectively.Through the method based on Bayesian formula to integrate the spatiotemporal information and depth information,we can improve the accuracy and robustness of detection,which is more efficient than traditional method.