Saliency Detection Algorithm For Images Based On Tag Semantics

With the development of the Internet and big data,it has brought a great deal of visual information.The number of images is larger and larger,and the visual features become richer.This series of change makes the development of computer vision faces greater challenge than before.Saliency detection,as an important branch of computer vision,plays an important role in object detection,semantic segmentation and autopilot.Among saliency detection algorithms,most of the traditional methods and deep learning methods only use visual features as image features;however,they ignored the important semantic information contained in image sematic tags.Therefore,the paper regards tags information as semantic feature besides visual feature,and combines visual feature with semantic feature to enrich feature expression of images and enhance image saliency region detection effect.At the same time,aiming at the problems of simple background and lacking of types in existing data sets,the paper collects a dataset based on tag semantic.The dataset brings some value and contribution to the saliency algorithm design based on tag semantics.The research and contributions of the paper are as follows:Firstly,we constructed an image data set based on tag semantics,which includes 2282 images in rail transportation area.The images are extracted from public transportation videos and collected from the Internet.Ground truth and semantic tags are used to label the images.Aiming at the imbalance of the collected images,we expanded the dataset by using the Generative Adversarial Network.Secondly,we optimizes convolutional neural network by merging shallow features and deep features.Saliency detection is a pixel-level task.After several convolutional layers and pooling layers,some spatial pixel information will lost,which is not conducive to the output of saliency detection.By comparing 3 different network structures and training conditions,we obtained the optimal structure and it is suitable for the saliency detection task in the paper.Thirdly,we proposed a saliency detection model based on tag semantic information.Based on semantic information and class activation mapping to detect salient regions of images,we used CRF model to merge the visual and semantic saliency results,and the outputs are optimized.We compared the results by using semantic features and only using visual feature,it can conclude that semantic information can make saliency detection more effective than before.At the same time,compared with the existing nine related methods in the public dataset and the constructed dataset,we found that the results are better than others in the three evaluation indicators by quantitative analysis.