The Research On Semantic-driven Image Mining Using Statistical Learning

Image mining is an interdisciplinary study for image analysis and understanding using data mining technology. It denotes the synergy of data mining, image processing, computer vision, image retrieval, match learning, pattern detection, database, and artificial intelligence. The fundamental task of image mining is to determine how low-level, pixel representation can be efficiently and effectively processed to identify high-level spatial objects and relationships. Broadly speaking, image mining deals with the extraction of implicit knowledge, image relationship, or other patterns not explicitly stored in the image databases. This dissertation tries to study the Semantic Gap problem in image. A framework of image semantic mining is developed, which includes the studies of image semantic hierarchy, image semantic object extraction, and semantic similarity measure. Based on those works, an XML-driven semantic-based image retrieval prototype is developed.As an extended research case, image semantic mining and service is part of "Theories, Mechanisms and Methods for Resource Organization and Management in Web Computing Environment: Agent-Based Grid-Enabling Service Organization and Management", which is the sub-project of National Grand Fundamental Research 973 Program of China under Grant No.2003CB317000.The main contributions of the dissertation are as follows:1. A FMMs-based image semantic hierarchical model is studied to bridge the Semantic Gap.According to the different semantic granularity, a four-level semantic hierarchy, which includes Image Blob Level, Meta-Semantic Level, High Semantic Level and Semantic Category Level, is defined to describe the image contents. Meta-Semantic in the hierarchy is used for a bridge between low-level character and high semantic. Based on the Finite Mixture Models (FMMs), the mapping relations between semantic levels are constructed. An optimized EM algorithm, which can return the best model structures, is used to estimate the parameters of FMMs. The experimental results of the hierarchical semantic classification prove the validity of the proposed model.2. An optimized algorithm, named HAB, is proposed to improve the performance of training process based on the idea of Boosting method.By defining a thorough evaluating function, the weight of each example in training set can be updated efficiently and effectively. And the misclassified examplesin the last time training will be set higher values. It means that in the next recurrent training, those examples will be focused to learn. At the same time the weights of positive examples, which have been classified correctly, will be evaluated to avoid the over-fitting problem. The experiment between AdaBoost and HAB proved that the proposed method has low train error and high robust.3. A new image semantic object extraction method using HAB algorithm is presented.The concepts of character-dense region and character-sparse region are defined. The image blobs, which are extracted from the character-dense region of the selected training image, are used as the feature template for the corresponding semantic object. Other blobs extracted from the remainder of training images are matched with feature template, and the results about the positive and negative sample information are achieved as the training data in the Feature Pool. Using the proposed HAB algorithm, the recurrent training process based on the Feature Pool improves the performance the image detector, and the results demonstrate the availability of the work.4. Based on the idea of Kernel Feature Set and Aided Feature Set, the image semantic measure is defined as an integrated method to measure the similarity of images. As an example, six semantic categories of natural scene images are presented, which are described with nine main meta-semantics. Using the proposed measure method, the image semantic categories are described statistically.5. XML-driven image semantic retrieval experimental system based on some ideas of our work, such as image semantic hierarchy, semantic object learning and semantic similarity measure, is realized. As shown in the primary results of experiments, those new ideas improve the performance of retrieval system.