Research On Mahalanobis Distance Based Metric Learning Algorithm And Its Applications

With the development of the national economy, our country is gradually carrying out industrial intelligent upgrade. Machine learning is one of the basic theories in intelligent industry. At present, most machine learning algorithms require measuring the similarity of samples which are described by feature vectors. Thus, metric learning, which do special study of similarity measurement function, has become one of the most important research topic in machine learning area. There are many kinds of distance measure functions. The Mahalanobis distance has become one of the most popular metric functions because of its excellent properties, including decoupling and dimension independent.Therefore, the research of Mahalanobis distance based metric learning algorithms has attracted many scholars’ attention. And the research achievements have become more and more abundant. However, there still exists some problems to be solved in the metric learning theory. This thesis will put forward three different metric learning algorithms to solve different problems, which further improves the metric learning theory. At the same time,this work also applies the proposed metric learning algorithms to some new applications,providing new methods and ideas to solve these problems.The first chapter introduces the background and significance of the study, and analyzes the principle of the metric learning. Meanwhile, the chapter also introduces the research status of metric learning, and analyzes several the state-of-the-art metric learning algorithms. These analysis helps find out the deficiency of the current metric learning theory, and put forward the research content.In the second chapter, the possibility of the combination of metric learning and support vector machine classifier is discussed. A support vector machine based Mahalanobis distance kernel learning method is studied. Firstly, this chapter makes a comprehensive analysis and comparison of K nearest neighbor classifier and support vector machine classifier, and discusses the relationship between these two classifiers and metric learning.Secondly, the Mahalanobis distance based radial basis function kernel is constructed, and the properties of the kernel are analyzed and compared with other kernel functions. Thirdly, this chapter proposes a metric learning algorithm using Mahalanobis distance based radial basis function kernel, which integrate metric learning and the support vector machine learning into one framework. At the same time, this chapter proposes a compressed representation method, which can improve the efficiency of processing high dimensional data. In addition, this chapter introduces the strategy of directed acyclic graph to solve the problem of multi-class data. The proposed algorithm in this chapter combines metric learning and support vector machine classifier, and effectively solves the difficult problem of kernel function selection for support vector machine. Experiments on the benchmark data sets demonstrate the high accuracy and stability of the algorithm.The third chapter discusses how to establish a universal applicable metric learning model in practical application. At first, this chapter studies the properties and physical meaning of logdet divergence. Then, the properties and performance comparison of different training constraints are also illustrated in detail in this chapter. And the comparison confirms the superiority of triplet constraint. On the basis of these research, the chapter proposes a logdet divergence based metric learning algorithm using triplet constraint. The algorithm improves the classification performance as well as reduces the conservatism. This chapter also uses a compressed representation method to deal with high-dimensional data. The method reduces the computation time a lot at the price of a little loss of accuracy. Another innovation of this chapter is the dynamic triplets building strategy. The strategy can choose the most useful triplets in each Mahalanobis distance training iteration cycle. All these work makes the proposed metric learning algorithm has a good performance as well as strong applicability. The experiment results on benchmark data sets prove the superiority of the algorithm.The fourth chapter researches the metric learning of instances with dynamic features.First of all, this chapter carries on the discussion of time series. And it makes a comparison of current similarity measurement methods of multivariate time series. Secondly,this chapter proposes a Mahalanobis distance based dynamic time warping algorithm to measure the similarity of multivariate time series. Besides, this chapter presents a metric learning model for multivariate time series. The algorithm can effectively learn the Mahalanobis distance which used in Mahalanobis distance based dynamic time warping measure. This chapter firstly uses the metric learning algorithms to learn and classify samples with dynamic features. Experimental results illustrate that the algorithm outperforms the classical methods to some extent.The fifth chapter applies the proposed three metric learning algorithms into new applications. The first application is the facial expression recognition. The chapter puts forward a metric learning based facial expression recognition framework. The framework uses behavior units to describe the geometric features of facial expressions, and uses the proposed three metric learning algorithms to learn Mahalanobis distance function to measure similarity of the behavior units. In the second application, this chapter presents a metric learning based image retrieval framework. In this framework, the pyramid histogram of visual words method is used to extract the appearance features of image. The Mahalanobis distance is obtained using the metric learning algorithm present in the third chapter. Then it can get the retrieval results by comparing the Mahalanobis similarity of testing image and images in the library. The third application in this chapter is the data-driven based fault detection. A metric learning based fault detection framework is proposed in this chapter. The framework constructs multivariate time series pieces from training signals. And it uses one-class metric learning algorithm to learn Mahalanobis distance function and the corresponding threshold. Measuring the distance between multivariate time series pieces from testing signal and fault-free training multivariate time series pieces, it can judge if the testing signal has any fault. The experimental results demonstrate the superiority of these proposed frameworks.