Research On Sports Video Content Analysis Using Player Behavior Information

With the rapid development of the technologies of computer, network and multi-media, there is an explosive growth in the amount of available multimedia information.Video is one of the most important components of the multimedia data, which has hugequantity and complex structure. As an important genre of video document, sports videohas attracted increasing attention in automatic video analysis due to its wide viewershipand tremendous commercial potential.The research of this dissertation focuses on the problem of broadcast sports videocontent analysis. To solve the problems of low-level video features cannot represent hu-man high-level semantic concepts, this dissertation proposes a novel approach for thesports video analysis based on the player behavior (trajectory and action) and the inte-gration of audio analysis in terms of semantics and tactics. Some important technologiesand solutions are studied which especially concentrate on the player trajectory trackingand action recognition, semantic/tactic mid-level representation construction using playerbehavior information and multimodal fusion with domain knowledge and semantic/tacticcontent analysis of broadcast sports video based on mid-level representation. The detaileddescription of the research content in the dissertation is as follows:A new player detection and tracking approach in broadcast sports video using sup-port vector machine and particle filter is proposed. Support vector classification combinedwith playfield segmentation is employed to automatically detect the players in sportsvideo. Then, an improved particle filter called support vector regression particle filteris proposed as the player tracker by integrating support vector regression into sequentialMonte Carlo framework. The improved particle filter not only enhances the performanceof classical particle filter with small sample set but also improves the efficiency of trackingsystem.A novel player action recognition approach in broadcast sports video based on sup-port vector machine and optical ?ow analysis is proposed. Different from the existingappearance-based methods, our approach is based on the motion analysis and extract mo-tion descriptor in terms of spatial distribution and grid partition of the optical ?ow fieldwithin the player figure region. In the proposed approach, the optical ?ow is treated as the spatial patterns of the noisy measurements instead of the precise pixel displacementsto enhance the robustness of motion descriptor. Support vector machine and temporalvoting strategy are employed to recognize the type of player action in the video clip. Theproposed motion descriptor and the action recognition approach significantly outperformsthe existing appearance-based method.A novel multimodal approach of highlight ranking for sports video summaries in af-fective context is proposed based on player behavior information and audio keywords ofsports game. The mid-level representation"trajectory-action-audio"is constructed for thevideo content by fusing the information of player trajectory, action and audio keywords.Based on"trajectory-action-audio", the computational affective features are extracted todescribe the objective process of highlight ranking of sports video summaries from usersubjective perception. A kernel based nonlinear probabilistic ranking model constructionmethod is proposed, which is robust for the noisy data and provided with good expan-sibility. In addition, a new subjective evaluation criterion is proposed to guide modelconstruction and feature extraction with the assistance of forward search algorithm.A new tactic analysis of broadcast sports video is proposed based on player trajec-tory information. Tactic analysis of sports video aims to recognize and discover tacticpatterns and match strategies that teams and individual players used in the games. Basedon players and ball trajectories, an algorithm of local temporal-spatial interaction analysisis firstly proposed. Using the multi-object trajectories, a weighted graph is constructedvia the analysis of temporal-spatial interaction among the players and the ball based onthe metrics of distance and shape in temporal interval and velocity and linking distanceamong intervals. The aggregate trajectory which is a new tactic representation of sportsvideo is computed based on the weighted graph. The interactive relationship of aggregatetrajectory with the hypothesis testing for trajectory temporal-spatial distribution are em-ployed to discover the tactic patterns in a hierarchical coarse-to-fine framework for theattach events of soccer game video.