Research On Real-time Video Stitching Technology And System Implementation

Video stitching is a technology that uses multiple small-field-of-view videos to build large-field-of-view videos.This technology is an extension of image stitching technology,but it is different from image stitching.With the development of information science and computer technology,video stitching has also been more and more widely used.Firstly,the basic process of video stitching,including video frame decoding,frame preprocessing,frame registration alignment and video frame image fusion is introduced by this paper.Then,the main technology involved in the video stitching process,including the lens imaging mathematical model,wide-angle and fisheye lens distortion correction algorithm,image spherical mapping and bilinear interpolation are elaborated in detail.For the problem of video registration,this paper introduces two types of common registration algorithms.It includes direct pixel registration algorithm and registration algorithm based on feature point matching,and the scope of application,advantages and disadvantages of the two types of registration algorithms are compared.For the case of multichannel video stream registration,in order to eliminate the ghost and blur caused by the accumulation of registration errors,this paper introduces a bundle adjustment method that optimizes the camera parameters globally.For the case where the input video parallax is serious,this paper introduces a parameterized motion model estimation method based on homography transformation field.Finally,for the specific situation of multi-camera system video registration,this paper proposes a method that combines direct pixel registration and SURF feature registration and globally optimizes the parameters of all cameras based on the bundle adjustment method.For the problem of video blend,this paper introduces several classic static image blend algorithms,and discusses the new problems in the process of video frame fusion compared to static image blend.For the problems of seams and blur caused by moving objects in video frames,this paper proposes a video blend algorithm based on optimal seam updating and convolution pyramid.The optimal seam update strategy and the idea of seamless blend of the convolution pyramid can avoid moving objects passing through the structure and reduce or even eliminate the seam and blurring effects in the result image.On the premise of high computational efficiency,a result image with good visual effects is generated.Finally,this paper introduces the design and implementation of a video stitching system based on multi-camera.The system uses a pipeline architecture to maximize CPU utilization.For the synchronization problems caused by the different processing speeds of different pipelines in the pipeline,this paper proposes a design of a buffer queue using mutual exclusion locks and condition variable protection which ensuring the calculation efficiency and the correctness of the data under the pipeline architecture is maintained.In view of the shortage of CPU computing resources for high-resolution input images,this paper proposes an implementation method that uses GPU to accelerate the frame processing algorithm based on data parallel;in the implementation of convolution pyramid,this paper proposes a implementation method using shared memory which makes full use of the computing and storage resources of the GPU and improves the computational efficiency of video stitching.Based on the above system,this paper conducted a five-channel 3840 * 2160real-time video stitching experiment,and the results of the experiment were in line with expectations.