Surveillance Video Coding With Background Modeling

In recent years, as the increasing requirements for emergency investigation, socialsecurity surveillance and smart video analysis, video surveillance systems are more andmore widely deployed in modern society. However, the long-time (months, even years)captured surveillance video also produces great challenges for video coding technology.On one hand, the tradional hybrid prediction/transform block based coding cannotsatisfy the requirement of high-efficiency surveillance video coding. On the other hand,there is much desire for low-complexity coding and transcoding algorithms to supportmultiple-path and real-time surveillance video coding. Inspired by recent developmentof model based coding and the characteristics of long-time static background ofsurveillance video, this paper engages to propose high-efficiency and low-complexitysurveillance coding algorithms based on novel low-computational-cost backgroundmodels.The main contributions can be summarized as follows:1. In order to remove the background prediction redundancy, this paper proposesnovel background models and updating algorithms for high-efficiency surveillancevideo coding. Based on a theoretical analysis result that using background modeledfrom original input frames for background prediction can perform better video codingefficiency, this paper firstly makes an in-depth analysis of coding efficiency, time andmemory cost of multiple background models. Following these, two low-complexitybackground models are proposed for high-efficiency surveillance video coding. In themeantime, a scene-content and encoding-parameter adaptive background updatingmodel is proposed to tradeoff between reducing bit-cost of background coding and theprediction efficiency of input frames. Experiments show that, based on the proposedbackground model based framework, the novel background models can respectivelyachieve1.19~1.23dB and0.91~0.99dB gains respectively in relative low and lowestcomplexity, and the adaptive background updating model can further achieve0.3~0.4dBgains. Moreover, experiments on videos with kinds of complex environments prove that,our method can apply to different weather conditions with non-weak lightness. 2. To reduce the prediction redundancy of the blocks mixed with background andforeground pixels, this paper proposes coding unit adaptive motion compensationsbased on background difference prediction. For block-based hybrid coding, this paperfirstly proves that there is still room to improve the motion compensation efficiency forblocks mixed with background and foreground pixels theoretically. Furthermore, wederive three conditions when coding the current block in background difference domain,namely background difference motion compensation (BDMC), can significantlyimprove the coding efficiency. Based on the analyses above, we build up a backgroundbased adaptive motion compensation(MC) model, which adaptively selects short-termMC (traditional MC using recently decoded frames as reference), background referenceMC (with high-quality encoded and reconstructed background frame as long-termreference), and BDMC for different coding units. Moreover, a specially designed fastmode selection model is adopted to select different MCs for difference macroblockclassifications. Results show the method has18.45~31.75%total bit saving，0.61~0.74dB foreground gains.3. To realize a further background redundancy reduction for the emerging videocoding standards HEVC and AVS2, this paper develops a hierarchical coding basedreference selection and bit-allocation optimization algorithm for HEVC and AVS2.Based on an analysis of their hierarchical coding structure in low-delay configurations,two components for efficiency improvement are concluded: hierarchical referenceselection and hierarchical quantization parameter (QP) decision. Inspired by this, weanalyze how to make use of the modeled background frame to optimize the emergingvideo coding standards’ coding efficiency for surveillance videos. Meanwhile, we alsosummarize how to optimize the QP relationships among frames and coding units forsurveillance videos. Following these conclusions, we assign different reference frameselection and QP calculation algorithms to improve the video coding efficiency fordifferent kinds of hierarchical prediction groups of frames. To reduce video codingcomplexity, we further employ a CU-classification based fast coding model to speed upsurveillance video coding. Extensive exepriments show that, this method respectivelyachieves44.8%and13.8%bit-saving for surveillance and conference videos, and thecomplexity reduction is more than40%.4. In order to make the proposed surveillance video coding techniques moreapplicable, this paper proposes macroblock classification based transcoding methods forpractical surveillance video transcoding systems and presents the proposed surveillancevideo coding techniques for AVS surveillance group. In this paper, we propose to speedup the coding and transcoding procedure based on following model: classifying input coding units into three different categories and applying specially designed models ofmotion estimation simplification, reference frame selection and candidate predictionmode calcuation for each category. Results show the proposed method achieved60~80times speed up than the H.264/AVC reference model. Moreover, we also integrate thisalgorithm into real-time surveillance video coding and multiple-path paralleltranscoding systems for AVS and H.264video coding standard. The related encoder andtranscoder are adopted by some companies in their products. In addition, the relatedtechniques in this paper are proposed to AVS workgroup through tens of proposals. As aresult, AVS surveillance group can save half the bitrate of H.264/AVC and AVS+videocoding standard.