Research On Image Feature Learning Based On Fisher Kernel

In the age of "data explosion", the problem of how to classify and filter the data effectively and further acquire valuable information that match the requirements of users, becomes a major problem we faced. In all types of data, image is the most common one. It contains a large amount of information and is easy to understand. Thus, analysis and understanding about image is becoming a hot research field. Specifically, this field mainly includes image classification, image retrieval and object detection etc al. These research tasks not noly have different research contents and purposes, but also be associated with each other.Due to large data volume and unstructured characters of image data, many operations, such as classification, can’t be carried out directly on original image for image analysis and understanding in most cases. So, image needs to be represented by a feature vector that is easy for handling. The quality of image feature representation impact on image analysis and understanding results directly. Image representation method has developed from digital image processing to feature learning. Unlike the former which generate features manually, feature learning executes supervised or unsupervised machine learning algorithms on a given data set to obtain the needed feature representation.Among most feature learning methods, Fisher Kernel utilizes a Gaussian mixture model to structure codebook for image local features, and gets global feature representation by computing log-likehood gradient get high-level feature representation on basic image features. Comparing with other models, Fisher Kernel has the advantage of representing image comprehensively and discriminatively. Meanwhile, as a standard feature learning framework, Fisher Kernel has a huge potential of compatibility and scalability. Therefore, the research about image feature learning based on Fisher Kernel is very important.Firstly, this thesis proposes a multi-scale and multi-codebook image feature representation to obtain better image features. This method introducts the concept of multi-codebook to increases distinction degree among visual words and utilizes the fact that different scales of image carries different information to improve basic Fisher Kernel. In addition, based on convolutional neural network which developed in recent years, we purpose cross-convolutional-layer pooling Fisher Kernel for image representation. It fully utilizes intrinstic properties of convolutional network and fuses advantages of both methods to get new feature representation. The contributions are as follow:(1) For basic Fisher Kernel model, single codebook is inefficient and contains a lot of useless visual words. To address this problem, this thesis proposes multi-codebook for jointly encoding, and utilizes differences among codebooks to provide more comprehensive feature representation. Meanwhile, we combine different scale of image with different codebook as input. Finally, we establish a complete multi-scale and multi-codebook system for image representation which further enhances the representation performance.(2) We combine convolutional neural network with Fisher Kernel. Convolutional neural network is used to extract image features while Fisher Kernel encode them to obtain final image representation. In this process, according to the characters of convolutional network, we propose multiple spatial units to extract more complete regional features and cross-convolutional-layer pooling to replace the traditional spatial pyramid and provides more accurate positional information. These improvements all make contributions to final image representation.