Research On Visual Saliency Detection For Stereoscopic Videos

With the rapid development of multimedia technology,multimedia information,such as images and videos,is growing.It has become impossible to handle such massive amounts of information manually.Therefore,the concept of saliency detection is proposed so that the computer can quickly capture the most interesting parts from a large amount of pictures or videos as human eyes,and then these parts can be prioritized and focused on.This paper focuses on the construction of a saliency computation model for stereoscopic videos.The existing 3D video saliency detection models always uses three types of information: 2D static information,motion information,and depth information.Most of existing works on 3D saliency are based on the assumption of fine quality of depth maps.However,depth maps from stereo matching or range sensors are usually with holes and artifacts,which severely drop the performance of those 3D saliency models.In addition,in current fusion methods for multi-feature saliency maps,many methods that use simple mathematical operations cannot effectively mine the effects of different features in different scenes,so that some scenes will have better results and some other will have poor results.This paper has studied the above issues.First,a cluster-contrast saliency prediction model is proposed for depth maps.The depth map is firstly segmented into several super-pixels,and the global contrast is computed with the centroid of the largest clusters of each depth super-pixel,and thus those bad effects originate from holes and other artifacts in depth are then eliminated.Finally the background prior information is introduced to reduce the influence of some natural scenes such as the sky or the ground have the same depth value with the salient objects.Then,a motion saliency map is calculated for each frame of the video using the dense optical flow.For 3D scenes,in order to detect the movement of the object on the direction of vertical to the lens,we added the depth information into 2D video's optical flow to obtain the displacement of each pixel on three directions: horizontal,vertical and vertical to the lens.Superpixel segmentation is also performed on motion maps,and a global clustering is performed to find the relatively static “background” super-pixel.And then we calculate the distance between other super-pixels with the "background" super-pixel to handle some scenes with camera moving.Finally,a fusion algorithm based on Bayesian formula is used to fuse the three saliency maps obtained by the three types of features.So that the different types of saliency maps can be mutually corrected,and combined with each other.The experimental results demonstrate that our saliency model has better performance of accuracy and robustness than other state-of-arts models.