Distance Metric Learning:Algorithm And Application

In the field of machine learning and pattern recognition,researchers often use distances between samples to estimate their similarity.In complex task scenarios,traditional distance metrics,such as Euclidean distance,Mahalanobis distance,Hamming distance,etc.,are difficult to truly represent the interrelationship between data.Therefore,distance metric learning(metric learning)has become an active topic in machine learning community.Metric learning automatically correlates data from training data,in order to make similar samples become close and dissimilar samples far away in the new feature space.In global metric learning,a metric matrix can be regarded as a linear transformation to the data.Considering that eigenvectors are the directions along which linear transformation occurs only by scaling,whereas eigenvalues are the scales along those directions.If there is a raw distance matrix,we may be able to improve its performance by fine-tuning its eigenvalues.Based on this idea,we proposes a new distance metric learning framework based on eigenvalue fine tuning in this paper.In the framework,we first get a rough distance matrix and then fine-tune the eigenvalues by optimizing the triplet loss to improve its performance.Under this framework,we propose new algorithms in global metric learning,local metric learning and dimension reduction.Theoretical analysis and experimental results proven the effectiveness of this framework.In addition,we propose another local metric learning algorithm based on quasi-linear kernel.Using this kernel,we are able to transform the metric learning problem into a classification problem of kernel support vector machines.On the other hand,in the field of deep learning,in order to improve network performance,the network is getting deeper and deeper,and the computing resources occupy and consume time are increasing.In order to reduce the consumption of computing resources and speed up the forward propagation speed of the network,network compression has gradually become a hot topic in the current research field.In order to make the network small and fast,researchers put forward a variety of optimization algorithms.These algorithms can effectively reduce and accelerate the model.At the same time,the faster and smaller model will face the problem of loss of information and performance degradation after compression.In order to reduce the impact of compression on network performance as much as possible,the network algorithm of teachers and students is gradually reproposed.The so-called network algorithm of teachers and students means that the network before compression serves as a teacher,and the compressed network acts as a student.Instruct the teacher network to guide the students to study online and improve their generalization ability.Most of the existing teacher-student network algorithms are devoted to students' network to learn a feature space close to that of the teacher network.However,when the expression ability of teacher network is much higher than that of student network or the data distribution is too complex,student network is hard to learn the same feature space as teacher's network.In this case,the teacher network may not able to help the student network.Sometimes,the teacher network even mislead the student network.Inspired by the idea of metric learning,we no longer focus on directly learning the similar feature space,but rather consider the matching among sample pairs.A new algorithm based on metric learning is proposed.This new algorithm is committed to ensuring the relative relationship between samples in those two feature spaces,expanding the space of feasible solutions,and improving the performance of student network.