Study Of 2D/3D Image Saliency Prediction

Visual attention is an important machinism of human visual system,which will selectively process the most important part of visual information and ignore other parts.Image saliency prediction is an algorithm that imitates such mechanism and thus is a common preprocessing tool in image processing/computer vision area,which is widely used in object detection and recognition,image quality assessment,and video coding,etc.In this paper,we focus on the study of 2D/3D image saliency prediction.The main research achievements are as follows:Firstly,we proposed a perceptual-based 2D image saliency prediction algorithm.The algorithm can be organized into three stages: estimation,activation and combination.Firstly,an orientation selectivity based local feature and a visual Acuity based global feature are proposed to jointly predict candidate salient regions.Subsequently,we introduce a visual error sensitivity based operator to activate the meaningful salient regions of saliency map from a local and global perspective.In addition,an adaptive fusion method based on free energy principle is designed to adaptively assign weights for mutiple feature channels of different images.Finally,we combine the sub-saliency map from each image channel to obtain the final saliency map.Experimental results on various natural and emotional datasets demonstrate the superiority of the proposed method compared to other state-of-the-art algorithms.Next,we propose a self-attention generative adversarial network for RGB-D saliency prediction.Firstly,we drive long-range dependency modeling and adversarial training to extract heterogeneous features from RGB and depth.Further,we design selective fusion module and prior initialization technology to efficiently learn the salient cues of intra-and cross-modality on RGB-D image pairs.Selective fusion module is used to adaptively select and fuse features from different modality in different level,and prior initialization is introduced to reduce the hungry demand for annotated RGB-D data and accelerate the convergence of model training by reusing RGB-based prior weights.Experimental results on common 2D/3D datasets demonstrate the superiority of the proposed method over other state-of-the-art algorithms.