| In recent years,with the advancement of hardware technology and the development of computer vision theory,virtual reality technology has also been brought up.In order to allow people to have a better immersive experience when watching live video,panoramic video live shows have become more frequent in our daily lives.Panorama technology has been widely used in areas such as virtual reality and navigation.The traditional video live program has a smaller viewing angle and can only be limited to scenes within a single point of view.Today,people's pursuit of visual experience is becoming higher and higher,and they hope to enjoy a more immersive experience.In view of this,related research on the panoramic video live broadcasting technology has become increasingly urgent.At present,the main problem of the panoramic video live broadcasting system is that the real-time performance and the quality of the panoramic picture are difficult to be taken into consideration.In view of the above problems,this thesis designs the various aspects of the overall system architecture and successfully achieves it.The system collects multi-viewpoint information of a scene by means of a multi-shot panoramic camera,and then stitches them into a panoramic video.After processing such as encoding and encapsulation is completed,the live streaming is distributed to a remote terminal device by means of a streaming media server.This thesis mainly did the following work:(1)A panoramic stitching algorithm based on multi-camera system calibration is proposed.In order to optimize the correction effect of fisheye distortion and improve the accuracy of image stitching,a multi-camera system calibration method based on feature descriptor calibration plate was used to calibrate the panoramic camera;the internal parameters of camera calibration and Mercator projection are applied to solve the problem that fisheye image distortion can not be directly used for stitching;in view of the high requirements of real-time performance of the live broadcast system,the SURF algorithm is used to extract and match feature points of adjacent images.Experiments show that the algorithm produces a good panorama.(2)Build a transmission network for live broadcast systems.For the problem that the existing network conditions cannot directly transmit the panoramic video,the AAC standard and the H.264 standard are used to encode the audio and video data,and the time stamp is used to synchronize the audio and video to ensure the synchronism of the terminal during playback.The above processing is based on development of the FFmpeg library.Consider the application scenario of live broadcast,the client/server transmission mode is constructed,and a server is used to send live streaming to multiple terminals.The SRS streaming media server is set up to realize real-time and reliable data transmission using the RTMP protocol.(3)Realization of live stream receiving and playing of remote terminal.The use of FFmpeg to complete the reception and decoding of live streaming;due to the poor interactive nature of the traditional live broadcast,a player based on Unity was developed to realize the change of the viewed scene with the rotation of the human head,which greatly improved the system interactivity and enhanced the immersion of the scene.This system ensures the good panoramic picture quality,and introduces the multi-threading mechanism to accelerate the speed of stitching,aiming at the real-time demand of live broadcast system.After a series of tests,the system's latency is about 5 to 6 seconds,and the number of frames played per second is close to 27 frames,which has a high practical value. |
PDF Full Text Request