Research On Surveillance Video Summarization Method Based On Spatiotemporal Analysis

The rapid development of video surveillance systems has produced massive surveillance video data,which brings great difficulties to storage.Furthermore,relying on manual methods to accurately screen and lock retrieval objects from hours of surveillance video is like looking for a needle in a haystack.Video summarization can present the valuable content in the long-term video in a concise form,which provides an effective solution for massive video storage and retrieval.Existing summarization methods are mostly based on the spatial information of video frames and analyze all pixels,which cannot take into account the information in the spatiotemporal joint domain.The computational efficiency of the methods is low,and the summarization display effect is not good.Therefore,this thesis carries out the research on the surveillance video summarization methods based on spatiotemporal analysis.The innovative achievements and the major research contents are as follows:(1)A surveillance video summarization method of adaptive object deformation in the spatiotemporal domain is proposed.Aiming at the problems of poor presentation of video summarization in the spatiotemporal joint domain and the deformation of spatiotemporal objects,this thesis proposes an adaptive deformation restoration algorithm for the spatiotemporal object starting from the sampling mode of spatiotemporal slices.The proposed algorithm uses the slope of the spatiotemporal trajectory to obtain the velocity of the object and uses the velocity value to determine the sampling width of the vertical slice where the object has minimal deformation,then adaptively restores the appearance of the spatiotemporal object.The proposed algorithm is extended to the multi-object scene.The average velocity value of the objects is used as the sampling criterion,and multi-column sampling is used to identify the moving direction of the object with transparency,and then the video summarization is constructed.The experimental results show that the proposed method can effectively solve the deformation problem of spatiotemporal objects,the generated surveillance video summarization is concise and compact,and the object contour is smooth and flat.(2)A spatiotemporal joint method for surveillance video summarization via the direction information is proposed.Aiming at the problem that the moving direction of the object is difficult to determine in the spatiotemporal joint domain video summarization,a spatiotemporal joint method for surveillance video summarization via the direction information is proposed.This method first uses the horizontal slice to obtain the object spatiotemporal trajectory;Secondly,the spatiotemporal trajectory background is eliminated and the linear trajectory slope is calculated,and the object motion direction is determined according to the object spatiotemporal trajectory slope;Thirdly,the motion segment in the sampling domain is detected to determine the timing position of the object in the video;Finally,the video summarization is constructed adaptively according to the object timing position and motion direction.The experimental results show that the proposed method can determine the motion direction of the spatiotemporal object with a low time cost,and the generated video summarization is concise and efficient.(3)A surveillance video summarization method based on spatiotemporal slice and dual attention mechanism is proposed.In order to further generate dynamic video summarization that is easy to watch,a surveillance video summarization method based on spatiotemporal slice and dual attention mechanism is proposed.Firstly,a kernel temporal segmentation algorithm based on the spatiotemporal slice(STS-KTS)is proposed for the accurate segmentation of the original video.The algorithm reflects the video scene information as spatiotemporal slice texture information,and uses the horizontal mapping method to project the preprocessed spatiotemporal slice into a one-dimensional array as the input feature of KTS;At the same time,taking the dual attention mechanism and grouping convolution as the basic components,combined with Bi LSTM,the spatiotemporal feature extraction network is constructed to quickly extract rich spatiotemporal feature information,so as to eliminate the excessive dependence of the existing summary model on a single feature with the texture feature information;Then,the frame parameter prediction module is used to obtain the best video frame contribution score,center score and frame sequence position;Finally,the frame scores are converted into shot scores to select content-rich segments and generate dynamic video summarization.The experimental results show that,compared with the existing methods,the proposed method has obvious advantages in the accuracy of generating abstracts.