Research And Application Of Distance Metric Learning Algorithm

Artificial intelligence is supposed as one of the most possible technologies to ignite the fourth scientific and technological revolution.Machine learning,as one of the theoretical cornerstones of artificial intelligent,is widely concerned by researchers from various fields.As an crucial branch of machine learning,metric learning studies how to learn a suitable distance function to measure the similarity between samples,which is used in a broad spectrum of applications such as clustering,classification,recognition,feature extraction and so on.Among the recent proposed metric learning algorithms,the triplet-based metric learning models are one type of widely used ones due to their excellent performance.However,there are some problems to be solved in triplet-based metric learning algorithms,such as the unknown role of informative sample mining,too many constraints for the learned metric parameter,etc.To solve those problems,two learning paradigms are proposed.Then,we apply them into matrix-based metric learning and deep metric learning to improve their performances.Those researching topics are the hot ones,they have important significance of both theory and practice.The main works and contributions of this dissertation are summarized as follows:· Firstly,to alleviate the problem of large scale of triplet constraints of traditional tripletbased metric learning,we propose a new metric learning model named improved large margin nearest neighbor.First of all,the purpose of the classical triplet-based metric learning model is analysed,which is to keep the purity of the neighborhood of a query sample,i.e.,only similar samples are located within the neighborhood.According to the purpose,we find that there are some geometrical relationships between triplet constraints,and most of the triplet-based constraints are redundant.In order to make use of those geometrical information,we construct a new metric learning model which only considers the triplet constraints containing the nearest dissimilar samples.Thus,the scale of the triplet constraint set is reduced greatly.Due to the non-continuous objective,we propose a continuous proxy of the improved large margin nearest neighbor.Then,combined with kernel techniques,the kernel-based improved large margin nearest neighbor is formulated to learn the non-linear relationship between samples.Finally,the numerical experimental results show the superiority of the proposed metric learning method.· Secondly,by analyzing whether there is harmful geometric information in the triplet constraint set,we find that the metric learning model suffers from serious inseparable problem without informative sample mining.To alleviate this problem,we propose an algorithm called adaptive neighborhood metric learning.First of all,the discrimination principle of triplet constraints is analyzed,and we find that there are a special type of regions in the feature space of a linear metric learning model.We call this region as inseparable region.As long as the dissimilar samples fall into those inseparable regions,there must be at least one triplet constraint never being satisfied by the learned metric parameter.The probability of dissimilar samples falling into the inseparable regions increases with the area of the inseparable regions,while the area depends on the number of similar samples and their distance to the query samples.Therefore,we propose an adaptive neighborhood metric learning which can adaptively select those separable samples to train the model.By analyzing the proposed model,we find that LMNN and NCA are the special cases of it.Finally,extensive numerical experiments conducted on several data sets show the effectiveness of the proposed model.· Thirdly,we study the traditional matrix-based metric learning model,and find that the matrix-based distance function derived from traditional bilinear projection has too few leaned parameters,which would limit the learning ability of distance metric learning model.To overcome this problem,we propose a new distance function called k-duplication matrix-based distance function.The design of the distance function is inspired by our analysis on the mechanism of traditional bilinear model,which essentially uses a series of linear filters with rank 1 to manipulate the matrix data.By increasing the rank of linear filters to K,we propose k-duplicated bilinear mapping,which has more learned parameters.By using this projection,a new matrix-based distance metric function can be constructed.This function is a general one which can be applied into various metric learning algorithms to form their matrix-based counterparts.Thus,we apply the new proposed distance metric function to adaptive neighborhood metric learning and obtain the matrix-based adaptive neighborhood metric learning method.Then,a matrix-based class-dependent distance function is proposed,and matrix-based multi-metric adaptive neighborhood metric learning model is obtained.Finally,numerical experiments on several matrix data sets illustrate the effectiveness of the proposed methods.· Lastly,by analysing the informative sample mining methods used in deep metric learning,we find that deep metric learning also encounters serious inseparable problem.Commonly,serving for the same goal,the informative samples in different metric learning algorithms should be roughly the same,and the criteria of the informative sample mining methods should be also roughly the same.However,there are some contradictions between different informative sample mining methods,such as semi-hard sample mining and hard sample mining.However,this contradiction could not be explained convincingly,while it can be explained successfully by using the concept of inseparable samples.Therefore,we conclude that the deep metric learning also suffers inseparable problems.Then,we propose deep adaptive metric leaning(DANML)to solve the inseparable problem,by adding two extra parameters to adaptive metric learning algorithm proposed in the previous chapter.The two parameters are used to stabilize the neighborhood radiuses in the procedure of statistical gradient descent method.We surprisingly find that several state-of-the-art metric learning methods are special cases of the proposed DANML,such as Multi-similarity loss,N-pairs loss,Proxy-NCA loss,etc.Finally,the effectiveness of the proposed method is illustrated by performing DANML on the fine-grained classification tasks and large-scale image retrieval tasks.