Research On Second-order Statistical Modeling With Deep Features In Image Classification

Image classification is an important research direction of computer vision.It has good application prospects in the field of commodity recommendation and mobile payment and is also a key supporting technology in high-tech industries such as unmanned driving.With the rapid expansion of the image scale and the urgent need for the development of the artificial intelligence industry,it is increasingly important to obtain highly discriminative and robust image representation on large-scale or complex scene image.In view of the powerful feature learning ability demonstrated by the deep convolutional neural network(CNN)in recent years and the fact that the second-order statistics can make full use of the information contained in the feature itself with simple and efficient calculation,the thesis develops a second-order statistical modeling based on deep convolutional features.Since the second-order statistical information can further enhance image representation,two new high-discriminate and robust image classification methods are proposed.The main research contents and innovative work of the thesis are as follows:(1)A second-order statistical modeling image classification method(HLBP-CNN)based on multi-layer convolutional features fusion is proposed.HLBP-CNN is based on a single CNN architecture and uses a second-order statistical model to exploit convolutional features of different convolutional layers to construct three bilinear pooling models with feature-level addition,representation-level addition and feature-level stacking followed by dimensionality reduction.HLBP-CNN not only effectively uses the second-order statistics of the inherent information of different convolutional layers,but also provides more accurate and robust image representation.Extensive experiments on seven widely used benchmarks demonstrate that HLBP-CNN methods are superior to the counterparts(e.g.,B-CNN),achieving a competitive or better performance compared to state-of-the-arts on both fine-grained categorization and content-based image retrieval tasks.(2)A robust estimation of covariance method based on deep convolutional features is proposed.Covariance estimation modeling of deep convolution features can effectively improve the image representation ability.But they lie in two challenges.One is robust estimation of covariance in case of high dimension and small sample size.Another is high computational and storage costs caused by high dimensional covariance representations.To tackle above challenges,this thesis proposes a novel robust covariance representation withlarge margin dimensionality reduction for visual classification.Firstly,this thesis introduces two regularized maximum likelihood estimators(MLE)to perform robust estimation of covariance in case of high dimension and small sample size,which can greatly improve modeling ability of covariances.Then,this thesis presents a large margin dimensionality reduction method for high dimensional covariance representations.It doesn't only significantly reduce the dimension of robust covariance representations with considering their Riemannian geometry structure,but also can further enhance their discriminability.The experiments are conducted on texture classification,face recognition and fine-grained image classification tasks,and the results show our proposed method is superior to its counterparts and achieves state-of-the-art performance.