Research And Application On Bag Of Features Based On Spatial Fisher Kernel Framework

This thesis is intended for the problem of image matching in the engineeringapplication projects, and we make an intensive study of image classification andretrieval. Bag of Features(BOF) method has been acknowledged as the promisingapproach in the image classification and image retrieval due to its implicity andperformance in recent years. This method considers only frequency statisticsinformation of the local points. When we apply the Fisher kernel to BOF methods, theassignment of the feature points can be mapped in a soft way. As a result, theinformation of the means and standard deviations of the features point can be embeddedinto the final BOF vectors and can get an excellent performance. But both the twoalgorithms describe an image by statistic information of local features. They disregardall information about the spatial layout of the features, and have severely limiteddescriptive ability.In this thesis, we introduces Spatial Pyramid Model(SPM) to the BOF algorithmbased on Fisher kernel and presents a new representation of image. The newrepresentation benefits from the SPM, it can express spatial information of localfeatures. This thesis improves the two discriptor vectors-Fisher Vector(FV) and Vectorof Locally Aggregated Descriptors(VLAD) which are both based on Fisher kernelframework. We subdivide an image and compute histograms of local features atincreasingly fine resolutions. Then we concatenate the appropriately weightedhistograms of all sub-vectors. At last, we get two kinds of new vectors, Spatial FisherVector(SFV) and Spatial Vector of Locally Aggregated Descriptors(SVLAD). Throughsundividing and concatenating, it can append geometric layout information of localfeatures to the original vectors. We compare our two new representations with theoriginal descriptors and test on several image data sets. Experiment results show thatour spatial Fisher kernel framework outperforms the original framework in bothclassification and retrieval applications.