Research On Approach For Synchronization Of Multi-Camera Videos In Social Media

With the rapid development of social media,,there are a large number of multi-camera videos related to the hot event captured by users on the network.Because the time points of multi-camera videos captured by users are different,it is important to align multi-camera videos with the global timeline to analyze them,so that multiple videos are played in sequence according to the time when they were captured,called the synchronization of multi-camera videos.It is a key issue in the field of computer vision and is the base of 3D reconstruction,panoramic stitching,and target tracking.However,the current synchronization of multi-camera videos methods face many challenges due to the complex content and uneven video quality of multi-camera videos taken by users in the network.The current research improves the accuracy and robustness of multi-camera video time synchronization through the effective fusion of the features of audio modality and the features of visual modality.However,the existing multi-camera videos time synchronization method based on audio-visual feature fusion is limited by audio-visual events.The time synchronization accuracy of multi-camera videos with complex content is low,it is essential to improve the accuracy of multi-camera video time synchronization for 3D reconstruction,panorama stitching and target tracking.Researching on approach for synchronization of multi-Camera videos in social media,the thesis is supported by the scientific research project co-constructed by the scientific research and graduate training of Beijing Municipal Commission of Education,"Analysis of Cross-Media Data Analysis and Mining Based on Social Awareness".To address the shortcomings of current multi-camera videos time synchronization methods,based on the fusion of audio and visual features,the existing multi-camera video time synchronization method is improved to improve the accuracy and robustness of multi-camera video time synchronization.The main contents are as follows:(1)The current situation of multi-camera videos time synchronization and the related research methods are reviewed and summarized.Firstly,the multi-camera videos of time synchronization methods are summarized and are classified according to the features used for time synchronization.The existing multi-camera videos of time synchronization methods are mainly divided into three categories,multi-camera videos of time synchronization based on audio features,multi-camera videos of time synchronization based on visual features and multi-camera videos of time synchronization based on audio-visual feature fusion.Then the advantages and disadvantages of each method are analyzed.Finally,the current research situation of multi-camera videos time synchronization is summarized,such as the problems and challenges of existing research methods and the advantages of audio-visual features fusion to multi-camera videos of time synchronization,and the research direction of this paper is pointed out.(2)The current visual features with poor robustness and multiple time synchronization points cause problems that time synchronization of accuracy and robustness is low.Based on the fusion of audio and visual features,this paper proposes a cross-validation multi-camera videos time synchronization method that combines the similarity of audio and visual features to improve the accuracy and robustness of multi-camera videos time synchronization.This method innovatively uses the SIFT feature of video frames to extract key frames to improve the robustness of visual features.In the method of audio and visual feature fusion,it fuses audio features of similarity with visual features of similarity combining video quality.It is implemented with a nonlinear fusion function,so that the audio-visual fusion method has a wider application in multi-camera videos of time synchronization to solve the problem of multiple time synchronization points.In the time synchronization algorithm,a cross-validation method is introduced for global time synchronization,which further improves the accuracy and robustness of multi-camera videos time synchronization.Through experimental analysis on two public datasets,the proposed method is better than the existing methods using audio features,visual features,and audio-visual feature fusion in time synchronization accuracy.When the error tolerance is 40ms,the experiment results show that the proposed method improves the time synchronization accuracy rate by 14.6%over the existing methods using audio-visual feature fusion,and verifies the effectiveness of the proposed method.(3)Aiming at the problem that the existing time synchronization methods are greatly affected by outliers,this paper proposes to normalize audio and visual features fusion self-paced learning method for multi-camera videos synchronization.The specific innovations of this method are mainly reflected in the following two aspects.In the audio and visual feature fusion method,the improved SIFT based visual feature and audio chroma feature are fused.The similarity of audio feature is calculated by clustering,and we introduces normalized processing to effectively solve the problem of outliers that may occur when audio and visual feature fusion.The theory of self-paced learning is combined with the time synchronization algorithm.The objective function is optimized to make the time synchronization algorithm automatically sort and learn the videos according to the audio and video quality,and priority is given to synchronizing the videos with better audio and video quality.In order to solve the problem that the accuracy of time synchronization is affected by outliers,the purpose of improving the accuracy of the algorithm is achieved.Through experimental analysis on public datasets,this proposed method is better than the existing methods using audio features,visual features,and audio-visual feature fusion in synchronization accuracy.When the error tolerance is 40ms,the time synchronization accuracy rate is improved by 13.6%compared with the existing audio and visual feature fusion method.