Research On Key Frame Extraction And Clustering Algorithm Of Esports Short Video

In the past few years,short e-sports videos have developed rapidly and become one of the main content people watch in leisure time.Although many short video platforms have built recommendation systems to improve users' click-through rates.However,most of them tend to recommend videos based on user viewing records or with higher click-through rates,which may ignore specific ones.The content and category of the video cause the recommended video to be less targeted,and the video recommendation click rate is low.In this paper,a recommendation method based on video key frame extraction and multi-modal clustering algorithm proposesed and implements for MOBA(Multiplayer Online Confrontation)short e-sports videos.In this thesis,the key frame extraction algorithm and multimodal clustering algorithm of esports short video are studied.The main contents include the following aspects:(1)A key frame extraction algorithm based on target detection model for short video of esports is proposed.E-sports short videos usually are highlights,which can be regarded as a shot.I proposed to use the most informative frame as the key frame.This method uses YOLOv3 target detection algorithm to calculate the amount of information in the image,and use the most deteced frame as the key frame of the video.Finally,this method is compared with the key frame extraction algorithm based on K-means clustering algorithm,compare the clustering effect of the two sets of data,and get the conclusion that the key frame extraction algorithm based on the target detection model is better.(2)A clustering algorithm for e-sports short videos based on deep multi-modal subspace clustering network is proposed.Short video is usually regarded as a file form with multiple modal information such as visual,audio,text,and motion characteristics.In order to take enough video features into consideration,multi-modal feature fusion is needed.The deep multimodal subspace clustering network(DMSCN)used in the paper has the functions of unsupervised feature extraction,feature fusion and clustering.Considering the attributes of e-sports videos and people's viewing habits,key frame,host images,and host's hero image,skills image,and audio features are extracted as the multi-modal features used by the e-sports short video clustering algorithm.In the multi-mode fusion method,affinity fusion and spatial fusion were respectively tried,and the effectiveness of the clustering algorithm for the clustering of short e-sports videos was verified through multiple sets of experiments.(3)Proposed and implemented a short video recommendation system for e-sports based on multi-modal clustering algorithm.In this paper,I realized the visualization website,"AI watch E-sports" for e-sports data.The website mainly designs a user management module,a short video recommendation module and a competition schedule module.The short video recommendation module adopts the previously proposed video key frame extraction algorithm and multi-modal clustering algorithm,which can recommend short videos according to the user's behavior records and mutli-modal features.