| With the wide application of the internet and the development of modern information processing technology,video data presents an explosive growth.Most of the video data are unedited long videos.Due to the complex structure,rich content and uncertain duration of long video data,people have limited processing capabilities for massive long video data.In order to facilitate understanding,utilization,positioning or understanding the content of long videos,the most effective way is to segment the video first.How to automatically and effectively segment long videos with different contents and a large number according to the semantic information in the temporal sequence is an urgent problem to be solved and has important scientific significance.The methods based on Temporal Convolutional Network(TCN)have achieved good accuracy in time-series segmentation,but this methods always suffer from over-segmentation and can not obtain an ideal F1-score based on intersection-union ratio,so they are difficult to apply in real life.In addition,the current multi-stage architecture is affected by the overfitting which caused by huge parameters limits the number of stages of the multistage model,so that the performance of the model can not be further improved.Our studies are as follows:Firstly,aiming at the problem that TCN series methods are trapped in over segmentation,this paper analyzes how to capture temporal semantic information and fuse effectively,and proposes the semantic guidance module SG.This paper finds that the main reason for over segmentation is the lack of max-pooling layer in TCN,so it is difficult to capture sufficient semantic information,resulting in over segmentation.SG captures high-level semantic feature through feature downsampling,and obtains temporal semantic features with the same length as video sequence through upsampling.Experiments show that using the temporal semantic feature of SG output to guide the temporal convolutional network can greatly reduce the occurrence of over segmentation.In order to better realize the guidance function of SG,the output of SG and the output of TCN are fused through the feature fusion module based on channel attention.Secondly,aiming at the problem that the number of stage of multi-stage model is limited by the number of parameters,a lighter parameter sharing temporal convolutional network PS-TCN is proposed in this paper.Based on the multi-stage architecture,this paper deeply analyzes the role of each stage of the model in the multistage architecture,and divides the multi-stage architecture into three parts: rough prediction generation,refinement,and final prediction.The exploration find that multiple TCN owned by the refinement part jointly complete the same task.Therefore,in the refinement part,the parameters of multiple TCN are shared through the information sharing mechanism to greatly reduce the amount of parameters of the entire network.Extensive experiments show that under the same conditions,the PS-TCN proposed in this paper achieves higher accuracy than the state-of-the-art backbone network and reduces the amount of parameters by about 16%.Finally,in order to strengthen the correlation between frames in video sequences,this paper explores how to improve the temporal modeling ability of the model,and proposes a Video Speed Prediction module(VSP)based on self-supervised learning.VSP makes the model fully understand the video temporal information by simulating the variable speed playback of the feature sequence.This module randomly samples multiple outputs in the multi-stage architecture and predicts the generated sampling rate through a simple classification network,thereby forcing the outputs of each stage in the main network to have sufficient temporal structure information.Through a large number of ablation experiments,this paper proves the effectiveness of the above three parts for video temproal segmentation.On this basis,this paper makes contrast experiments of temporal segmentation on three public datasets Breakfast,GTEA and 50 Salads.From a large number of experimental results,it can be seen that under the same conditions,the accuracy,editing distance and F1-score of the model proposed in this paper greatly exceed the benchmark method and achieve state-of-the-art performance.This research will provide effective preprocessing for various video understanding research tasks,and has important theoretical value and practical application prospects. |
