Research And Implementation Of Multiple Video Streams Image Stitching System

Panoramic video surveillance system, which is the real-time transmission of multiple video stream and a mosaic for each frame image, has a wide range of applications in many fields such as medical composition, site monitoring, and so on, but the real-time playback of panorama video is still in the research stage. Video decoding, split-screen display and image mosaic are the focuses of this study. The split-screen display is about to receive the videos and play them in different locations on the same screen, and the image mosaic technology is a mosaic of several images, may be different time, different perspectives or different sensors,with which can be built a large seamless high resolution image. The image involves the problem of conversion, then we use the projection transform to process the image mosaic in this paper, and see the effect of the mosaic.In this thesis, we study the multi video streams based on local video and network camera, and we make a generalization of the existing real-time mosaic algorithms, and use the projection coordinate transformation method to pre process the image, and view the results of the mosaic, then the main work is as follows.(1) Firstly, we realize the functions of image with video decoding, video play and image processing, by using the FFmpeg library of video decoding, SDL library for video playback, and Open CV Library for image processing. On the basises of these, we do some experiments on the video streams of local video and video camera, play each frame of video decoding, and sharpen, blur, gray each frame of the video.(2) Secondly, we realize the functions of image feature extraction and feature points matching, mainly including the feature extraction of Harris corner, FAST feature points, SURF feature points, SIFT feature points and the feature points matching of SURF feature vector, SIFT feature vector, BRIEF feature vector, ORB feature vector. Then we use RANSAC algorithm to eliminate the feature points of matching error and obtain the accurate transformation matrix. We use Open CV to achieve the whole process of searching the feature points, matching the pictures, and image blending, finally display the processing effect.(3) Thirdly, we realize the function under different projection modes for image mosaic, including the two-dimensional plane, sphere and cylinder, and observe the effect of seamless splicing, and analysis of the efficiency of projection mosaic.In this thesis, we have realized the function system of the video image mosaic, including the video's split screen, and we have extracted and matched the feature points of the images.And we test and analyze different video mosaic algorithms, which has laid a foundation for the future of more complex video mosaic system.