Research On Surveillance Video Synopsis Based On Spatio-temporal Optimization

With the construction and development of smart cities,surveillance cameras are widely installed and applied,while the amount of video data generated presents explosive growth.How to efficiently utilize surveillance video is a current research hotspot and also a technical difficulty to be solved urgently.Therefore,video synopsis technology for the purpose of briefly expressing surveillance video has been received extensive attention.Video synopsis extracts and analyzes the object in the original video,and adopts the method of changing space for time to eliminate the redundancy among frames.However,the existing video synopsis methods have some problems such as inaccurate interactivity preservation,poor performance of video synopsis in complex scene and false collision caused by shifting moving object in the time axis.Therefore,under the National Natural Science Foundation of China,this thesis carries out the research on the theory and algorithm of surveillance video synopsis based on Spatio-temporal optimization and strives to achieve high-quality surveillance video synopsis.The innovative research results and major work completed are as follows:(1)An surveillance video synopsis method with accurate preservation of interaction among the moving object is proposed.In order to solve the problem of inaccurate retention of interaction among the moving objects in condensed video,a Spatio-temporal interaction algorithm was proposed,which divided the object tube into tube set by considering the Spatiotemporal proximity and vector correlation of the tube pairs in the original video,and constructed the tube set with interactivity.On this basis,a frame sequence fusion algorithm based on Spatio-temporal constraints is proposed.Frame sequences and frame vessel are introduced into the algorithm,and the tube set is optimized by continuously inserting the frame sequence into the frame vessel.The experimental results demonstrate that the proposed method can accurately preserve the interactivity between moving objects.(2)A surveillance video synopsis method based on the integration of multiple Spatial-temporal slices and the dichotomy is proposed.Most of the existing video synopsis methods are based on object detection and tracking to generate object tubes,which is prone to the problem of incomplete object trajectory and poor performance when applied to crowded and complex scene surveillance video.To solve the problem of incomplete objects trajectories when condensing surveillance video in complex scenes,this thesis proposes a surveillance video synopsis method based on the integration of multiple Spatial-temporal slices and the dichotomy,which proposes a Sparse-Crowed Video Synopsis(SCVS)framework,which divides the input video into sparse and crowded segments and processes them respectively.Firstly,a sparse and crowded clips segmentation method based on the integration of multiple Spatio-temporal slices and dichotomy is proposed to divide sparse and crowded clips.Secondly,the object tube is extracted from the segmented sparse clips,and the synopsis video in the sparse clip was obtained by optimizing the rearrangement of the tubes.Finally,sparse synopsis video and crowded clips are fused to generate a complete synopsis video.The experimental results show that SCVS framework can effectively avoid the problem of poor performance of preprocessing technology in a crowded scene,which has a higher frame condensation ratio and object preserving ratio,and comfortable visual perception.(3)A surveillance video synopsis based on Spatio-temporal offset is proposed.In order to solve the problems of false collision and chronological disorder caused by the object tube only shifting in the time domain,a surveillance video synopsis based on Spatio-temporal offset is proposed,which can migrate moving objects in the time domain and space domain simultaneously.In order to acquire the optimal temporal and spatial offset of the object tube,a trajectory optimization algorithm based on Spatiotemporal offset was proposed.Without Collision Relation and Collision Relation are defined by analyzing the space-time relation of object tube’s translation in time domain,and the corresponding spatial offset strategy is proposed for Intersecting Collision and Overlapping collision in Collision Relation.In order to acquire time-domain migration,a frame density model is proposed.Experimental results demonstrate that the proposed method is superior to the existing methods in improving condensation rate and reducing false collision.