Research And Application Of Panoramic Video Virtual Photography Technology

With the development of 5G technology,the VR product will become more and more important.As a display form of VR video,panoramic video provides users better viewing experience.However,due to the limitations of the human eyes,users can only watch the Field of Views(FoV)of the panoramic video.As a result,they cannot fully enjoy the video content and probably miss important imformation.Therefore,the virtual photography function can solve this pain point,which is implemented by automatically capturing the most interesting video area and guiding users to watch the Normal Field of View(NFoV).Aiming at the above problems,a panoramic video virtual photography system is designed and implemented in this paper.The main work is as follows:In order to capture the most interesting video content accurately,a saliency detection network for panoramic video based on spatio-temporal features fusion is proposed,which can convert the input panoramic video into saliency score maps through a encoding and decoding structure.The model solves the boundary stretching and image distortion problems of equirectangular video by cube mapping with paddings.It uses multi-scale feature extraction and feature fusion based on attention mechanism to obtain effective spatio-temporal feature information,and combines it with the decoding model to complete the prediction of the saliency score maps for each frame.The model achieved better accuracy than existing models on the public data set and verified the effectiveness of the model.Based on the research on viewing guidance of virtual photography,a content-aware NFoV path planning algorithm is proposed.Based on the saliency detection model and optical flow estimation,the video saliency area can be found.Firstly,the algorithm finds the best viewing guiding path around the most salient area.Then,it calculates a FoV-aware path that can display the most salient object and the surrounding environment more effectively.Finally,it generates a smooth and stable NFoV virtual photography video through path smoothing,and helps users watch panoramic video more comfortably.The comparative analysis with AutoCam and other algorithms verifies the performance advantages of the algorithm.The practical test results show that the system can generate smooth and stable NFoV virtual photography video and help users to obtain a better viewing experience.