Complementary Perceptual Saliency Detection Research

Saliency object detection is a technology of extracting saliency region(i.e.,the region of human interest)from image or video by algorithm.Among them,saliency object can be defined according to texture,color,size and even motion state.Salient object detection technology is helpful to other computer vision tasks and real life applications.At present,the task of salient object detection mainly relies on the feature learning of convolutional neural network to obtain more number and type of features.Thus greatly improving the detection performance.In this paper,convolution neural network is used to do exploratory and innovative research for salient object detection.Overall,our main contributions can be summarized as follows:(1)During the research process of image saliency object detection,we find that the saliency object detection model only obtains salient features from one dimension of foreground information and background information,without comprehensively considering and using these two kinds of information.Because the feature learning ability of deep convolutional neural networks depends on the ability of shallow networks to learn salient features.If the shallow neural network obtains certain features of the salient object,it can be passed to the deep layer to continue learning and optimization.In other words,this part of the feature is foreground information.And if the shallow neural network fails to learn some salient features,it is very difficult to obtain or restore these features in the deep layer.In other words,these features are "hidden" in the background information.In the end,the network detection results are incomplete due to the missing part of the significant object information.Considering that the attention mechanism model can optimize the saliency feature information obtained in the saliency detection network,we propose a complementary attention mechanism saliency object detection network.Our network includes two branches: a positive attention mechanism for detecting foreground information of feature maps and a negative attention mechanism for detecting background information of feature maps.We fuse the feature map output by the negative attention mechanism with the feature map output by the positive attention mechanism to make up for the incomplete results of the output of the positive attention mechanism.At the same time,in order to fully obtain multi-scale and multi-level features,we have introduced a two-way structure and multi-level supervision.Experiments show that the algorithm in this paper has a good performance in the evaluation index of salient object detection.(2)How to make full use of the characteristic information of salient objects has always been the focus of our research.We find that the different convolutional layers of the salient object detection convolutional neuraly network have different types of feature information.We also find that the feature information between different channels of the feature map in the same convolutional layer is also different.In response to this,we design a stacked U-shaped salient object detection network using the channel attention mechanism.This network includes a parallel hole convolution module and a multi-cascade channel attention mechanism feedback module.The function of the parallel hole convolution module is to expand the convolution receptive field without increasing the amount of calculation to capture more salient features.Our original intention for designing the multi-cascade channel attention mechanism feedback module is to make the salient features of different convolutional layers complementary,and to form the internal dependency of feature maps between different channels in the same convolutional layer.Furthermore,the feature information of saliency targets of different convolutional layers and channels can be better utilized.In addition,in order to fully supervise the learning of the middle layer of the neural network,we also adopt a multi-level supervision method based on the crossentropy function.Experiments show that the algorithm in this paper has excellent performance in the evaluation index of salient object detection.(3)The task of video salient object detection is an advanced task in the research of salient object detection,but also a difficult task.The video task not only needs to consider the salient characteristics of the object in the spatial dimension,but also needs to consider the dependence of the time dimension between frames.Therefore,it is necessary to comprehensively utilize spatial information and temporal information.In summary,we have designed a spatiotemporal complementary graph convolutional video saliency detection network model.The detection network can combine the spatial information in a single frame and the time information between multiple frames for comprehensive use.When extracting spatial information in a single frame,we use foreground and background prior knowledge and graph convolution theory,and fuse the extracted features into complementary information.When extracting the spatiotemporal information between multiple frames,we use a different information extraction method: LSTM,3D convolution and optical flow method.Instead,a new spatiotemporal attention mechanism model is designed.The model can learn associated features based on multiple frames of information in the same video.In addition,we also design a local two-way structure with the aim of fully learning the representation of salient object.Experiment show that the algorithm has excellent performance in the evaluation index of video salient object detection.In addition,the real-time processing speed of our model has reached 23 FPS.