Image Matching Via Deep Recurrent Neural Network

In recent years,image retrieval,person re-identification and face recognition have been applied to our daily life.Image matching which to judge the similarity of two images is the basic component for these technologies.While the goal of patch-based image matching is to judge the similarity of patches that extracted from image.Finding accurate correspondences between patches is instrumental in a broad variety of applications including object instance recognition,fine-grained classification,image stitching and multi-view reconstruction.In this paper,we focus on patch-based image matching based on deep recurrent neural network(RNN).We found that the human's vision system performs matching process in a recursive manner.While previous work on matching can all be regarded as a one-off solution which cannot evaluate the similarity of two patches sufficiently.Therefore,we propose a new matching algorithm to simulate human's matching process.Firstly,we propose a novel matching algorithm based on convolutional neural network(CNN)and RNN.Similar to common matching methods,our method using convolutional neural network to extract feature of two image patches.The difference is that we measure the similarity of two features by putting the features into recurrent neural network in a recurrent manner.Our experiment proves that the combination of CNN and RNN can simulate human's matching efficiently and the recurrent manner can explore the relation of two features more deeply than the one-off manner and obtain a more accurate similarity metric.Secondly,we introduce a novel monotonous loss based on cross-entropy loss to our new network.The monotonous loss can restrain the similarity metric of all nodes in RNN in a monotonous manner and make the prediction more and more accurate as the node goes deeper.Human's judgement is more and more confident as the time of observation goes up and monotonous loss is designed to make our network has the same property.Our experiment shows that the novel monotonous loss makes the network has monotonous property.Finally,we introduce online hard negative mining to patch-based image matching to solve the imbalance problem.In each iteration of training,we pick the hard negative examples of a batch to update parameters.Our experiment shows that online hard negative remits the imbalance problem to some extent and the performance of our network gets further enhance.