Research On Mobile Phone Image Stitching Based On Improved AKAZE-GMS And Grid Optimization

In recent years,with the rapid improvement of mobile phone image quality,mobile phone camera has gradually replaced professional camera as the main tool of people's daily photographing.Usually mobile phone camera is used to take landscape photos.However,limited by the size of the camera lens,the captured images can not always meet the needs of people for wide field of view landscape images.At this time,it is necessary to use image stitching technology to stitch the captured images.At present,almost all smart phones have already equipped with panoramic photograph mode.Mobile phone panoramic photograph mode has an instruction guide to guide the image shooting,which is easily affected by light conditions,motion conditions and other factors.Images captured in this mode has the characteristics of small parallax,small scale and rotation change,and high image resolution.The principle of image stitching algorithm in most mobile phone panoramic mode is the same as that of the AutoStitch algorithm,which is based on the single viewpoint transformation model.However,there are two serious problems in this kind of algorithm:first,this algorithm may fail to stitch for the scene with too large parallax,such as the scene that the shaking amplitude above and below is too large;secondly,the overall naturalness of the generated image is poor,the corners around the image are prone to distortion,and the internal linear structure is seriously deformed.To solve these problems,based on a large number of research and experiments,this paper takes the mesh optimization deformation model as framework and combine it with the binary feature extraction algorithm and fast feature matching algorithm,proposed a mobile phone panoramic image stitching algorithm with good stitching effects and certain real-time performance.The main research contents in this paper are as follows:(1)A feature extraction and matching algorithm based on AKAZE-LATCH-GMS is proposed.Based on the real-time requirements of mobile phone image stitching and through a large number of literature research and simulation experiments,we choose the faster and more stable algorithm of AKAZE-LATCH to replace SIFT for feature extraction.In order to further improve the speed of feature extraction and matching,an efficient and stable feature extraction and matching algorithm based on AKAZE-LATCH-GMS by combining AKAZE-LATCH algorithm with BF matching algorithm and grid based motion statistics method was proposed.Furthermore,in order to improve the accuracy of the algorithm,two improvements are made in the combination process:First,we associate the division of the triad pixel grid in the latch descriptor with the scale space parameters to improve the scale invariance of the algorithm;Secondly,we introduce the bidirectional matching strategy in the Brute-Force matching algorithm to improve the accuracy of feature matching point pairs.Simulation experiments show that,compared with the traditional SIFT+RANSAC feature extraction matching algorithm,the proposed algorithm not only greatly improves the computational performance,but also has better robustness to common scenes such as illumination,fuzzy changes and so on.(2)A mesh optimized image stitching algorithm with linear structure protection is proposed.In view of the poor stitching effects of the image stitching algorithm in mobile phone,we analyzed the NISwGSP algorithm which based on mesh deformation optimization model.To solve the problem that the algorithm lacks the protection of linear structure in the image,a linear constraint term is constructed and added into the model to solve the problem of linear structure deformation in the image.In order to reduce the influence of illumination difference on the final image effect,a weighted fusion method based on human visual sensitivity characteristics is used for image fusion.In addition,we also introduce the OpenMP multi-threading scheme into the algorithm and reduces the dimension of the model energy function to optimize the efficiency of the algorithm.Compared with several mainstream stitching algorithms,the improved algorithm performs well in most scenes.(3)An image stitching algorithm based on the improved AKAZE-GMS and mesh optimization is proposed.Combined the proposed feature extraction matching algorithm with the improved NISwGSP image stitching algorithm,we introduced a stitching algorithm for mobile phone panoramic image.Aiming at the problem that the mobile phone panoramic mode may produce stitching results with ghost and blur in large parallax and moving environment,we combine the stitching algorithm with the fusion algorithm based on the optimal stitching line,and provide two different fusion modes to deal with different shooting scenes.Experimental results show that the proposed algorithm can better stitch the image sequence captured by mobile phone,and the generated image has a visual effect.Through the subjective and objective evaluation of the generated image,the proposed algorithm has better stitching performance.In addition,the time-consuming comparison between the proposed algorithm and the AutoSitch algorithm show that the proposed algorithm can basically meet the real-time requirements.