Research On Salient Region Detection Methods And Applications Based On Visual Saliency

With the popularity of the Internet,the traditional ways of entertainment are no longer the main ways gradually.More and more people choose online entertainment and communication tools.Under this background,social network platforms have emerged.Image social has become the main function of social platforms,which has brought hugeamounts of social images.How to deal with social images quickly and effectively using limited computing resources has become a problem to be solved urgently.The main motivation of the paper is image processing pressure of social images.Taking saliency detection technology as breakthrough point,we choose salient region detection methods as the main research content and choose image classification task as application extension of method research.The work has very important theoretical significance and application value.Research contents and innovative results are summarized as follows.1.Aiming at the current situation of no saliency datasets for social images,a saliency dataset for social images is constructed.The source of images,principles of image selection,annotation of images and statistical analysis of dataset are discussed in detail.In order to verify the performance of the new dataset,the new dataset and the current seven popular datasets are evaluated.The experimental results show that the new dataset has the advantages of rich salient region sizes,high connectivity with image boundaries,no obvious central prior,small color difference of salient regions and images.The new dataset can be used for further saliency detection research.In particular,the new dataset has tag information and provides conditions for new salient region detection methods.2.The researches have showed that only low level features cannot get good salient region detection.And the sate-of-the-art methods fail to fully exploit tag information for saliency detection.So,a salient region detection method is presented which fully considers tag information and uses CRF model.The method combines the image appearance features with tag context information,which reduces the distance between high-level semantics and low-level features of the image.The experimental results show the importance and effectiveness of tag semantics for salient region detection.The proposed method is compared with the 23 popular detection methods.ROC curve,AUC value,F-measure value,PR curve are higher than all other methods.The precision and recall are improved.3.Deep learning is widely used in salient region detection.Although salient region detection results based on deep learning features are better than results based on handcrafted features,but there are differences on individual image.So,a salient region detection method based on multi features for social images is proposed,which uses both deep learning features and handcrafted features.Deep learning features include CNN feature and tag semantic feature.In addition,the results of classic handcrafted feature based detection methods are used as useful supplements to deep learning feature based detection results.The experimental results show that the detection results based on multi features are significantly better than methods based on only deep learning features or only handcrafted features.4.Image datasets are divided into scene datasets and object datasets based on whether they contain salient regions or not.For scene datasets,multi-annulus partition based pooling region selection and multi-words hard coding method are proposed.The combination of these two methods can provide fast classification for scene datasets.For object datasets,saliency based soft coding method is proposed,which not only highlights the importance of salient regions for image classification,but also reflects that local spatial constraint plays an important role for coding consistency.The experimental results show the effectiveness of the proposed method.In summary,saliency provides a new way for image classification.