Domain Knowledge And Deep Feature Fusion For Semantic Event Analysis Of Basketball Videos

With the rapid expansion of video information and the continuous strength of computing capabilities,video analytics based on deep learning has achieved good results in large-scale video data.However,most of researches are focused on video surveillance,which involved single person(or multiple person independently)motion analysis.In fact,the videos of multi-person coorperation motions are popular in real life,especially in team sport videos,such as backetball,football and so on.In these kinds of videos,there are usually the problems including compex background,fast objection motion and object occlusion.Therefore,it is a challenging issue in video analysis.In this thesis,we focus on the semantic event classification of basketball game videos.The principal works are as follows:First,we proposed ontology based global and collective motion patterns for event classification in basketball videos.A semantic event in broadcast basketball videos is closely related to both the global motion(camera motion)and the collective motion.A semantic event in basketball videos can be generally divided into three stages: pre-event,event occurrence(event-occ),and post-event.By analyzing the influence of different stages of video segments to semantic events discrimination,it is observed that different video segments are effective for different classification task.We generate a new dataset NCAA+ from the existed NCAA dataset.The proposed scheme includes a two-stage GCMPs based event classification scheme on Pre-event and event-occ and an image based success/Failure classification algorithm on post-event.The GCMPs are extracted using optical flow.The two-stage scheme progressively combines a five-class event classification algorithm on event-occs and a two-class event classification algorithm on pre-events.Both algorithms utilize sequential convolutional neural networks(CNNs)and long short term memory(LSTM)networks to extract the spatial and temporal features of GCMPs for event classification.Second,we utilize post-event segments to predict success/failure using deep features of images in the video frames based algorithms.Finally the event classification results and success/failure classification results are integrated to obtain the final results.To evaluate the proposed scheme,we collected a dataset called NCAA+,the experimental results demonstrate that the proposed scheme is effective.Second,we study player detection,segmentation and player posture estimation.based on deep learning.By Statistics,we found that players' postures vary in different events,so we further analyze individual players' postures.First of all,the locations of the players are obtained by using SSD algorithms.Then we propose a super pixel based FCN-CNN object segmentation algorithm.It can effectively remove the background and is helpful for the subsequent posture estimation.Finally,the player postures are estimated based CNN.Third,we propose a semantic event classification scheme by combining multiple features.First,we integrate the characteristics of the GCMPs with the statistical features of the player's postures to achieve multi-level semantic information.Then using LSTM to achieve the expression of fusion features in the time domain,and achieving the classification of basketball events.The final experiment proves that the proposed method using multiple features is more effective than GCMP based algorithm.Fourth,the video analysis system of basketball match is realized.In the Linux operating system,a complete basketball event classification system is implemented based on the MATLAB platform.