Research On Image Mmatching Algorithm Based On Heterogeneous Siamese Network

Image matching technology is one of the key technologies in the field of computer vision.It has a wide range of applications in various fields.Traditional image matching algorithm relies on the manually selected shallow features,which has poor adaptability to influencing factors,such as interference and viewpoint transformation.And it usually cost a lot of time in practical applications to select a target that is adapt to the feature of the algorithm.In this thesis,inspired by the powerful feature extraction ability of deep learning,we design a image matching algorithm based on heterogeneous siamese network.In this thesis,we design two deep neural networks that using Image matching technology based on the similarity comparison net firstly,and analyze the strong points and shortcomings of them.Secondly,amelioration is made to the fully convolutional siamese network which is similar to the conventional image matching algorithm in matching procedure.We replace the backbone with residual network18(ResNet18)firstly.Considering the asymmetry of network branch characteristics and metrics,yielding two heterogeneous siamese feature pyramid network are designed(HeS-FPNv1,v2)with the combination of lower-level detailed features and high-level semantic features.Moreover,depth separable convolution is utilized when the template is convolved with the detection image,which improves the real-time performance.Also,the multi-scale template of the full convolutional twinning network is abandoned,replaced by the regional proposal network(RPN)after the correlation conventional,which can automatically generate multi-scale matching results.Considering the image matching characteristics,we change the way to select positive and negative samples in the traditional RPN network,and the corresponding loss function is modified accordingly.Finally,the gradient constraint penalty term is added to improve the performance of the network.In order to further improve the matching accuracy,a heterogeneous multi-layer feature fusion siamese region rroposal network(MF-HeSRPN)is proposed,inspired by the siamese region proposal network with the bounding box correlation.So that the measurement of similarity and bounding box regression are simultaneously conducted.Moreover,a location refinement structure is attached to the end of network to improve the bounding box accuracy.Taking the property of the network in to consideration,the sample selection procedure of the RPN network is further improved.The harmonious loss function are utilized to reduce the influence of imbalance samples.The gradient constraint penalty term is also used to improve the network stability and enhance performance.Finally,we test all nets on the public dataset and the self-labeled dataset,which verifies the effectiveness of the improvement.The MF-HeSRPN achieves best result on test datasets,which demonstrates its noble generalization ability.It is also proved that matching accuracy can be profoundly improved by adding bounding box regression to the network.At the end of the thesis,we compared the deep learning approach with the traditional matching algorithm,which verifies that the deep neural network is much more robust to interference than the traditional algorithm,and also is adapt to other influencing factors such as viewpoint transformation.