Employment Of Deep Learning On Multi-spectral Image Descriptor Research

Image feature descriptor is a feature description for the neighborhood of image key point. Traditional manual descriptor can hardly describe nonlinear relationship among multi-spectralimages for their grey scale value, color, etc, varies greatly. Seeing the difficulty, we research on the construction of multi-spectral image descriptor with deep learning to improve efficiency of matching task among multi-spectral images.Actually, the essence of multi-spectral image descriptor generation with deep learning is learning the image feature, which find the feature space in which different spectral images from the same scene have shorter distance and vice versa. To achieve that, we design a multi-spectral image descriptor convolution neural network frame structure, which consist of a sampling module, a feature extract network module and a metric network module. The input of network is the anchor and the negative, positive patches. After 5 convolution layers, we extract the feature and generate descriptor with full connected layer while variables are updated by minimization of triplet loss. To speed up the convergence of network, we use matchnet to do the fine-tuning in training phase. Considering the loss of image structure information, we concatenate the output of lower strata and upper strata in the end of network and add Global context into descriptor. Simultaneously, we design and manufacture an on-board video caption system to capture large amount of multi-spectral images while getting ground-truth of multi-spectral image patches will involve great hardness. Till now, our multi-spectral dataset has reached 82,230 pairs. As demonstrated by experiments, descriptors generated by our model keep features and structure information of images. Additionally, the accuracy of matching task based upon our descriptors perform 42% better than SIFT descriptorsand 17.9%better than EOH descriptors.