Research On Image Semantic Information Extraction And Classification

Image semantic information extraction and classification approaches are the significant topics in computer vision area.With the rapid development of digital media technology,the semantic information extracting from multimedia data is important to analysis and understand the multimedia source.In the past few years,many image semantic information extraction and classification approaches achieve some progress in the aspects of visual feature generation,semantic model construction and strongly supervised learning manner.However,due to the semantic gap between the low-level image feature and the mid high-level semantic information,the existing image semantic extraction and classification approaches make slow progress in terms of object model learning,analysis of the correlation information,weakly supervised learning manner and discriminative feature construction.Therefore,how to design the effective image semantic extraction and classification models to address the above issues is an urgent problem.For this reason,this paper aims to conduct the exploratory research of image semantic information extraction and classification problem.The main research works and innovations of this thesis are listed as follows:1.One main drawback of existing image semantic extraction methods based on the local regions of image is that it fails to explore the local semantic information within image sufficiently.In addition,it is always required to perform a complex linear search of the parameter space for obtaining the ideal results.To circumvent this problem,we propose a discriminative sparse representation model to extract the semantic superpixels.Thus the semantic information extraction of local object regions in image is cast into the image decomposition problem.Specifically,the proposed approach designs a new discriminative regularization term and this new regularization constrain is combined with the reconstruction residual and sparse constraint to form an unified objective.Finally,we deduce an effective optimization algorithm to achieve the solution.The main advantage of our approach is that it can explore the local semantic information within image substantially and avoid the complicated process of parameter tuning.The proposed approach is an extension of current superpixel extraction method via constructing the object semantic model.2.In order to overcome the scarcity of semantic information from single image,we utilize the correlative information of multiple images and further propose a co-superpixel generation method based on graph matching.The main property of proposed method is that it is capable of capturing the consistent intermediate-level semantic information in images and is beneficial to guide the semantic information extraction of local image regions.In details,we treat the semantic extraction of image regions into the problem of superpixel merging and matching,then the correspondences of superpixels can be solved by the graph matching algorithm.In our method,we present a new superpixel merging criterion in terms of graph matching cost and the appearance similarity of adjacent superpixels.Based on it,the generated superpixels can capture the local semantic information within image effectively.The proposed method is the further development of current graph matching approaches based on image local interest points and provides a new way for generating the semantic regions of image.3.Based on the above semantic information extraction from image local regions,we further attempt to capture the semantic information from object regions within image.Object detection approach provides an important manner to capture the semantic information of image object regions.To address the problem of information starvation problem based on single image in the existing object detection methods,we propose to discover the consistent object-level semantic information from multiple images and promote the semantic information extraction power of object region.In particular,the proposed approach treats the semantic object detection into the problem of jointly discriminative dictionary learning and object window localization.Finally,the semantic extraction power from image object regions can be improved.Besides,most of existing object detection methods have the limitation that the trained object model applies only to a specific object class.For this reason,we propose an objectness detection approach to capture the object-level semantic information by combining the top-down and bottom-up visual dictionary learning.The main strength of proposed approach is that it can address the class-independent object detection task.The above proposed approach demonstrates a new scheme to extract the semantic information from object regions of image in terms of the association information mining and the applicability of object model.4.Among the semantic information extraction approaches based on object regions in image,most of existing methods require the heavy supervision or detailed label information to train the object model.In order to overcome this shortcoming,we propose a new semantic part learning approach via a designed discriminative clustering algorithm.Our main observation is that even though the appearance and arrangement of object parts may have the variations across the instances of different object categories,the constituent parts still maintain the underlying geometric consistency.Different from the current objection detection methods,our approach is capable of learning the object semantic parts with only the bounding box label.The proposed approach has the strong applicability that can discover the object-level semantic information across different object categories and generates the object regions with high semantic content for the problems of image classification and understanding.5.Based on the above semantic information extraction approaches via local regions and object regions in image,we further aim to construct the semantic representation of whole image and address the image classification.Since the current dictionary extraction methods neglect the representative visual samples can capture the discriminative semantic information for classification task,we propose a feature discovering approach incorporated with a wavelet-like pattern decomposition strategy to tackle this problem.In details,the proposed approach aims to exploit the representative visual samples from each class and further decompose the commonality and individuality visual patterns jointly.The property of our dictionary learning objective is that it can improve the discriminative power of feature representative.To further discover the semantic information with more discriminative power,we then integrate our visual dictionary objective into a waveletlike hierarchical architecture.Due to the hierarchical structure,the discriminative power of feature representation can be further promoted step by step.The proposed dictionary learning objective shows a new way to explore the discriminative semantic patterns from visual samples.6.How to extract the discriminative semantic patterns plays a critical role for image classification problem.However,the current visual dictionary extraction ignores to explore the relevance of dictionary atoms and weaken the semantic representation power of image feature.Therefore,we first propose to discover the sharable visual atoms for linking the different dictionary atoms via a novel discriminative class-specific dictionary learning objective.The proposed approach incorporates a consistency constraint of representation coefficients and the group sparsity regularization term simultaneously in a unified optimization function.With the aid of dictionary atoms with more sharable capability,the semantic information of image representation can be enriched.Besides,we also introduce a new dictionary learning model to explore the correlation of dictionary atoms by learning a over-complete dictionary and a compact dictionary jointly.The above dictionary learning objectives can promote the semantic power of image representation and provide an effective way to explore the relationship between visual atoms for classification problem.