Research On Methods For Local Feature Extraction And Description Of Images

With the development of information technique, the amount of image data and im-age category is rapidly increased. It is an important and urgent task to let computers automatically analyze and understand images. As the first step towards image analysis and understanding, feature extraction and representation of images are bases for address-ing all kinds of visual tasks such as image matching, classification, and retrieval. Due to the fact that local features are robust to background interference, occlusions and viewing angles, and can statistically provide a kind of image content representation, the study of local features is of importance. However, in the real world, images usually have many kinds of categories and complex structures and in most cases the imaging conditions are hardly available. With the coming of the era of big data, it is significantly challenging to effectively extract local invariant features and efficiently represent image contents.In this context, by starting from local image features and treating the effectiveness (discrimination and robustness) and efficiency (low computational complexity and low feature dimensionality) as the overall goal, this dissertation focuses on studying methods of extracting local invariant features, global image descriptions and efficient binarized feature representations. These methods have been successfully applied to feature-point matching, image classification or large-scale data retrieval. The study content and main contributions are summarized as follows.1. Since existing methods have low discrimination in feature description and weak robustness to noise, key-point descriptors based on local polar DCT (Discrete Cosine Transform) features and local second-order statistics with soft-pooling are respectively proposed. From the point of extraction of frequency features, the former first design-s polar-grid structures to sample image patches, and then leverages the classical two-dimensional DCT to extract frequency coefficients, followed by the regular scanning to rearrange the coefficients. These steps result in a low dimensional and robust local de-scriptor to image blur, noise and JPEG compression. From the point of extraction of high-order statistical information, the latter constructs a rotationally invariant local pattern set and proposes the soft-pooling based on the gradient order. By further using covariance matrix as the second-order statistical representation, it effectively improves the feature discrimination and robustness.2. Since the traditional LBP (local binary pattern) has limited power in spatial description and weak resistance to noise, image descriptors based on hierarchical fea-tures and local noise-robust patterns are respectively proposed. The former combines the wavelet transform and LBP to construct feature representations from the pixel level, patch level to image level. Due to the soft quantization and higher-order statistical coding of semantic patches, this descriptor is shown to be suitable for classifying images with spe-cific objects and complex scenes. The latter first uses different mean-valued difference masks to compute rich feature responses, and then performs the global threshold-based quantization and coding. These steps result in a low dimensional, rotationally invariant and noise-robust texture representation.3. Since LBP has weak descriptive power for complex textures, by starting from the construction, quantization and encoding of local features, three types of texture de-scriptors based on joint statistics of local X-patterns are respectively proposed. Firstly, a texture representation using an ensemble of binary codes is proposed. By introducing multi-orientation and multi-type filtering and the binarization operators, the descriptive power of LBP for the neighborhood is effectively enhanced. Secondly, a texture rep-resentation based on the joint statistics of local quantized patterns is proposed. This representation utilizes local quantized binary and ternary patterns to encode the texture information in image and gradient domains in a hybrid way. It reduces the computational complexity and feature dimensions of the first method. Lastly, a texture representation based on space-frequency co-occurrences of local patterns is proposed. This representa-tion employs two-channel multi-resolution spatial filtering and local Fourier transform to encode the space-frequency features in a rotation-invariant way and improves the texture classification accuracies.4. By taking into account both the effectiveness and efficiency in feature representa-tion, a texture descriptor using locally encoded transform feature histogram is proposed. Specifically, the multi-scale extremum filtering is proposed by taking advantage of s-teerable filters, thus providing a theoretical basis for computing multi-scale rotationally invariant responses. The operator of feature transform is further proposed to construct local feature set with both the discrimination and complementarity, and the ratio and uni-form quantizers are designed to obtain the discrete texture codes. Also, the adjacent-scale and full-scale coding schemes are proposed to generate the compact histogram features. This method is practically shown to be efficient, low dimensional and robust to changes in term of image rotation, illumination, scale, viewpoint and even with noise.5. Due to the fact that there are very limited labeled data and a large number of unlabeled data available in practical applications, a semi-supervised manifold-embedded hashing method is proposed to binarize traditional real-valued features. This method integrates manifold embedding, feature representation and classifier learning into a joint optimization framework. A robust loss function is obtained by adopting the l2,1-norm. A two-stage optimization strategy is proposed to effectively solve the hashing learning problem. The proposed method has achieved our desired goal for the task of large-scale data retrieval.