Video Salient Object Detection And Application Based On Spatiotemporal Gradient Fusion

Salient object detection is to use the computer to simulate the visual attention mechanism of the human eye,and extract the most important parts in a large number of redundant image contents or video data to facilitate the later image processing,such as segmentation,scaling,tracking,and so on.Video saliency detection is performed by frameby-frame detection or the addition of motion features based on image saliency.Usually,the video saliency model cannot detect the significant target in videos with complex background.In this thesis,video salient object detection algorithm with spatiotemporal gradient is proposed.Spatiotemporal gradient is obtained by combining the static edge and the dynamic edge.According to the similarity of the adjacent frames,the weights of the two frames are adaptively calculated to obtain the motion saliency map.The final saliency map is obtained by integrating the saliency map and the static image saliency map.The main work of the dissertation includes:First,we use spatiotemporal gradient to locate the salient object.The edge of a single static edge or motion information cannot accurately highlight the edge of the object.The dynamic information obtained by the large displacement motion optical flow and the static edge obtained by Canny edge detection is adaptively fused to get the spatiotemporal gradient,which can effectively suppress the invalid edge information in the background and extract the significant object contour accurately.Secondly,aiming at the problem of ignoring space-time consistency in the existing video saliency algorithm,time consistency method is introduced.Time consistency means that the significant value of the foreground/background area of the continuous video frame should change steadily along the time axis rather than abruptly.Through the constraint between adjacent frames,we can further refine the edges and remove background noise to get the motion saliency map.Finally,by adaptively determining the fusion weights,the static saliency map removes excess background noise while the motion saliency map avoids salient object missing,and the final video saliency map is close to the Ground Truth.The saliency detection results of our method are applied to video scaling and video segmentation.The video scaling experiments were carried out with content-aware video scaling techniques;salient regions in the saliency map were taken as image foreground prior to segment foreground of the video.The algorithm is compared with nine typical image and video saliency detection algorithms on two public datasets ViSal and SegtrackV2,using subjective evaluation and objective evaluation method including PR curve,F-measure,and mean absolute error(MAE,Mean Absolute Error).The experimental results show that the algorithm proposed in this thesis can extract the significant object in the video sequence more quickly and accurately,especially when the background is complex.In the application of video scaling based on content aware and video segmentation,the algorithm is more universal and superior than the other four video saliency detection models.