Visual Feature Representation Based On Detailed Spatial Relationship Information For Image Classification

At present, the image classification has become an important research topic in computer vision and pattern recognition, and also has extensive applications in the real situation. Image classification technologies can perform automatic recognition and understanding by extracting the key features from the images, and further provide effective solutions to solve practical problem. Visual feature extraction and representation are the most basic and important procedure in the image classification algorithm. Moreover, effective feature extraction and representation methods can reduce the dependence on the follow-up machine learning algorithms, and also have deep influence upon the performance of the whole classification system.Abundant spatial relationship information can be found in the images, accurately describing the spatial relationship information could effectively across or narrow the semantic gap between the low-level visual feature and high-level semantic, which has important significance in improving the performance of image classification. At present, many research works focus on describing spatial relationship information of the images. However, the spatial relationships among image primitives might be very complex under some specific application backgrounds such as medical images. The detailed spatial relationships are directly associated with semantics, and a minimal error or deviation may result in wide divergence. The existing correlational researches on spatial relasionship representation are difficult to meet the above demand. To this end, we focus on studying accurate description of the detailed spatial relationship. Based on the detailed spatial relationship information, we propose to construct some effective feature representation methods to further improve the image classification accuracy. Meanwhile, we also combine some specific domain data to perform extensive research, such as medical, scene, and remote sensing image datasets. The main works and contributions of this thesis are summarized as follows.(1) We propose a new topological relationship model named DTString. Most current topological relationship models can represent basic spatial relationships with limited numbers, but cannot capture the details of spatial relations. Thus, they cannot fully satisfy the actual demand of visual feature representation. The proposed DTString model based on string description can accurately describe the full details of topological relationships between image regions. DTString is further proved to be an atomic relation model. Thus, DTString has complete description ability for topological relations, and undividable topological relations can be captured with it. DTString is called the detailed topological relation model. Moreover, DTString-based reasoning algorithms are also proposed. These reasoning algorithms can provide some topological properties, such as reverse relations and sub-region types, by using purely string-based reasoning and without any geometric calculations, which lay a good foundation for effective similarity measure.(2) We propose a novel image classification method based on detailed topological relations. More specifically, we study several similarity matching algorithms such as exact matching, mirror matching and part matching based on proposed DTString model. Then, we apply the DTString model and similarity matching algorithms to geometrical structure retrieval. Next, we study two sub-region sampling strategies(i.e. regular grid and super pixel sampling) in the image classification framework using local features. We further propose a new image classification algorithm based on DTString and hybrid sub-region sampling. The experimental results of geometrical structure retrieval and scene image classification demonstrate that our DTString model and similarity matching algorithms can effectively describe the detailed spatial information of image feaure primitives and further enhance the discrimination ability of the visual feature.(3) We propose a novel texture descriptor named SAHLBP by studying the local detailed spatial relationships among the texture primitives. Most existing texture descriptors mainly describe the frequency of local patterns in the image, but ignore the spatial relationships among local patterns. These spatial relationship information provide crucial feedback in the process of distinguishing complex samples(especially for samples that emphasize details), such as medical images. To address this problem, we propose a novel method to improve local binary patterns descriptor by describing local detailed spatial relationships between the texture primitives. Firstly, we present an adaptive neighborhood radius search algorithm to determine texture primitives. Next, we propose a spatial adjacent histogram strategy to encode the local spatial relationships among the texture primitives for image representation, and then SAHLBP descriptor is obtained. Compared with the existing methods, the proposed SAHLBP descriptor has two significant advantages:(a) it can effectively describe local spatial relationships between texture primitives and provide more powerful description ability;(b) it has scale invariant property, i.e. it is insensitive to scale and resolution change. Finally, we introduce a classification framework based on SAHLBP and apply it to medical image classification. The experimental results on four real medical datasets show that our method can achieve outstanding performance in medical image classification. We also evaluate the impact of different parameters, classifiers, and frameworks on classification performance for medical images.(4) We propose a new spatial encoding method by studying the spatial relationships among the local features. Most existing methods mainly build fixed spatial template or divide space quadrant to count occurrence of local features as spatial relationship features. These methods still suffer from the problems of insufficient feature representation and weak discrimination ability. To this end, we consider kinds of spatial relationships of local features, such as direction, co-occurrence, and distance relationships, in the bag of visual words model to perform effective spatial encoding and learn deep spatial relationship patterns. More specifically, we first define a selection method of relevant key points based on distance and frequency, and then we propose a novel local structure feature that encodes the spatial attributes between a pair of points in a discriminative fashion using class-label information. Next, we learn a bag of structural words model as mid-level image features to obtain image structural representation. At last, we present a unified image representation by combing appearance and structure features. Experimental evaluations on four benchmark datasets demonstrate that the proposed method can achieve good performance in scene image classification.In summary, in this paper we mainly study the feature representation based on spatial relationship information among different image feature primitives, and a series of corresponding methods have been proposed. The experimental results demonstrate that performing accurate description of detailed spatial relationship information can effectively enhance representation ability of visual feature, and further improve image classification performance.