The Study Of Point-line Fused 3D Reconstruction And Deep Siamese Network-based Image Recognition

3D reconstruction as a key task in computer vision is always an important research direction which can promote the development of relative research and application fields.Efficient reconstruction that is based on image pairs and about big complex scene is still a hot spot and also challenging research,which can be applied to many actual applications,such as robot navigation,commercial manufacture and urban modeling.Current recon-struction based on point matching method is already ripe,while higher level line-based and semantic reconstruction is still complicated and in the stage of immaturity.In view of the above and with the consideration of the fact that a large scene usually contains large scale of points and lines,Some researches including line matching,camera and 3D point estimation,speedup of bundle adjustment are quite worthy.With the rapid development,deep learning which is represented by convolutional neural networks has attained significant results in im-age recognition,image segmentation and object tracking,and so on.These researches and applications would obviously boost the semantic reconstruction.This dissertation focuses on some research problems related to image-based recon-struction.These problems include line matching,point and line fused reconstruction,au-tomatic differentiation,positive matrix system solver and siamese network-based classifier.The primary research contents and contributions are as follows.(1)Focused on the higher level line matching problem which is more complicated than feature point matching,an efficient,fast and robust line matching method is proposed based on homography and optical flow tracking.In this method,images(include reference image and query image)are transformed by global homography or homography grid,and the similar areas are extracted.Lines that obtained by using LSD algorithm in the reference image are sampled,and tracked on the query image by using optical flow method.Then,based on the similarity between images,the tracking points corresponding to sample points can be used for fast line matching under a localization strategy.To speed up the estimation of the homographies of grids,a”rank-one”update method of SVD is utilized.(2)To solve the problem of accumulative error in incremental SfM and the problem of unified reconstruction of point and line matching,A point-line fused global SfM pipeline is proposed.In this global SfM,linear programming method is used to optimize the L_?errors of absolute directions for all cameras.When estimating the absolute positions of cameras,the relative positions of camera triplet are estimated first.Then camera triplets are fused and optimized by using linear programming.In order to obtain more line matches between images,a graph weighted by point matches is formed for unordered image set.Ant colony algorithm is utilized to find an optimized sequence in this graph.(3)An efficient parallel automatic differentiation method is proposed specifically for computing large scale sparse Jacobian matrix in bundle adjustment optimization.This method provides concise C++style writing of projective function,and automatically per-form the computation of reprojection and Jacobian matrix with high speed.When writ-ing projective function,C++operator overloading is employed to generate computational graph,followed by the generation of forward and reverse computational sequences.We attain reverse sequence by using topological sorting due to the dependency relationship between nodes.For large scale bundle adjustment,cross platform OpenCL framework is adapted to execute reprojection and derivatives on CPU or GPU according to forward and reverse computational sequences.(4)For the Levenberg-Marquardt algorithm and Dog-Leg algorithm used in the bun-dle adjustment,large scale positive definite matrix system should be solved,within which OpenCL-based parallel Cholesky decomposition and triangular matrix system solving are needed.In such a solving problem,small matrix sub-blocks is used to maximize the use of high speed local memory and register in computing device,which can accelerate the com-putation of inverse matrix and matrix multiplication.The inverse of sub-blocks is reusable in solving triangular matrix system.(5)We propose an efficient classifier for image recognition application based on siamese network and spatial transformer network.On the basis of convolutional neural network and ReLu activation function,two identical deep networks that share the same parameters are constructed.This two-way network serves as a classifier through similarity training by us-ing criterion and training sample which is augmented by geometry transformation.Adding spatial transformer network in front of the siamese network to rectify the input sample can significantly improve the accuracy of recognition.By using theories and techniques in the fields of computer vision,deep learning,nu-merical and parallel computation,methods of efficient point-line fused 3D reconstruction and deep siamese network-based image recognition are proposed and implemented in this dissertation,which are constructive and referential,and can enrich the theory study and practical application of 3D construction.