Research On Visual Saliency Detection Method Based On Multi-type Hybrid Features

Video saliency detection(VSD)improves the analysis and processing efficiency of videos by simulating the human visual attention mechanism.Due to the complexity of video scenes,such as blur,occlusion,camera motion,multi-objects,and objectwise interaction,VSD is more difficult and more challenging.The dominant technical difficulty of VSD lies in how to fully understand the semantical logic,motion feature and model the temporal relationship,thus to reasonably calculate the salient regions of each frame in the form of deduction to meet the temporal consistency.Based on the above discussion,this paper focus on two crucial issues of spatiotemporal feature(STF)extraction and time series(TS)relationship modeling.Firstly,for complete STF extraction,this paper discusses the advantages and disadvantages of the offline motion prior and the end-to-end two-stream STF extraction algorithms based on the full understanding of semantic association,and creatively proposes a spatial saliency guided local motion feature extraction method.Explicit motion feature calculation not only helps to improve detection accuracy,but also enhance the interpretability of the method.Secondly,for TS relationship modeling,this paper proposes a short-term and long-term TS modeling method,which deduces the saliency of the current frame from both local and global perspectives.Two scales of time windows help to learn multiple-scale temporal correlations in the pasted sequence of current frame,which is advantageous for obtaining saliency predictions that take both spatial smoothness and temporal continuity into consideration.Finally,a consistency modeling method is proposed to explicitly define the inter-frame local similarity specification and the global salient object enhancement protocol to further ensure the global consistency of the salient regions.Qualitative and quantitative experiments on five publicly available benchmarks with 26 state-of-the-arts demonstrate the effectiveness of the proposed method and prove that this research provides a new feasible solution for VSD.