Research On Salient Object Detection For Multi-Grained Annotation

With the recent rapid growth of 5G and social media,huge mount of annotation images naturally become the most direct media for transmitting the visual information.How to enable the computers understand the semantics in multi-media data effectively and accurately has become the main trend in the field of artificial intelligence and computer vision.As one of the fundamental research in computer vision tasks,salient object detection has attracted much attention in the community.This task presents great difficulties subject to the expensive acquisition cost of large-scale pixel-wise image annotations,as well as due to the challenges carried by image scenes(e.g.,occlusions,blur,large object-deformations)and the inherent complexity of human visual attention behavior.Therefore,this thesis designs a series of salient object detection algorithms combining with the corresponding machine learning theory,and it puts more emphasis on exploring the salient object detection with multi-grained image labels.Our main contributions can be summarized as follows:1.With the pixel-wise annotations,this thesis mainly focuses on how to improve the information propagation ability of network with the perspective of enhancing features from single-level.Existing feature pyramid network based salient object detection methods usually propagate information between adjacent levels at fusion stage,and they ignore feature communications from long distant levels,hindering from well preserving the beneficial information from high-level and low-level features.This thesis develops a novel and simple architecture,Cross-layer Feature Pyramid Network(CFPN),which dynamically aggregates and distributes multiscale features to improve the FPN ability for salient object detection.Comprehensive experimental results illustrate that CFPN establishes new state-of-the-arts on multiple benchmarks.2.Based on the above study,this thesis further improves the information propagation ability of network from the perspective of enhancing aggregation features from multi-level.Thus,for the aggregation features from all levels,this thesis first proposes a new concept of dense attention to efficiently control the information propagation.Then it proposes a novel attention Dense Attentive Feature Enhancement(DAFE)module for efficient feature selection and enhancement in salient object detection.DAFE consists of several couples of efficient attentional units,e.g.,Feature Refinement Unit(FRU)and Dense Attentional Collection Unit(DACU).The former one learns an attention map for selecting features from all of its subsequent FRU,while the later one densely correlates a set of attention maps from all its previous FRU to enhance feature from current FRU.Experimental results demonstrate the superiority of the proposed DAFE over the state-of-the-arts.3.For incomplete labeling,existing deep learning based salient object detection methods are limited to insufficient training samples and unsatisfied results(i.e.,saliency maps with incomplete object structures and blurring object boundaries).This thesis represents the salient object detection as an ordinal regression based multilabel ranking problem,and it proposes a novel cost sensitive label ranking method for weakly supervised salient object detection.Concretely,this algorithm utilizes saliency maps generated by several unsupervised salient object detection methods as the imprecise annotations.By mining the correlation between these labels,it learns a robust saliency classifier to distinguish the salient region of the test image.This method not only reflects the characteristics of multi-class classification,but also integrates the advantages of regression model,and it does not need large scale pixel-wise annotations as saliency labels,which overcomes the problem of insufficient large-scale labeled samples requirement under fully supervised framework.4.For a group of images without any labeling,this thesis mainly studies the cosaliency detection by exploring the intrinsic correlation between multiple images.Existing co-saliency detection methods simply consider the visual similarity among consistent salient objects during the phrase of feature matching,this ignores the consistency relationship in structural space.Thus,this thesis introduces the graph matching theory and proposes a novel graph-matching based model for co-saliency detection.Concretely,the proposed method casts the co-saliency detection as a graph matching problem,where the common salient object for multiple graphs is identified via integrating feature correspondence,intra-saliency coherence,and spatial continuity simultaneously,and the initial intra-saliency is iteratively refined via seeking optimal graph reconstruction.