Research Of Structured Algorithm For Video Synchronous Comments

With the rapid development of the Internet,the number of users in online video is growing explosively.Watching and commenting on video has become an essential part of users’ daily life.In interaction with video,there emerges a new interactive mode: synchronous video comment(Danmaku).The presentation of Danmaku is similar to rolling subtitles suspended on the video screen.This interactive way can significantly enhance the user’s shared viewing experience and provide data support for content providers and other decision-makers(such as advertisers,investors,retailers,educators,etc.)to understand real-time video content and audience feedback.However,Danmaku is relatively short,and there are still many gaps in related exploration.At present,the relevant research constantly analyzes the text as a standard text,not fully considered its characteristics.At the same time,the combination of the actual scene is not close enough.In fact,as a supplement of video content,video synchronous comment text contains the emotional views and understanding of users in the related video content,effectively alleviating the complexity of the large-scale image and video modeling and assisting the video content analysis.The synchronous review has its time synchronization properties and language characteristics,so the existing text processing technology is challenging to apply.Therefore,it is of great significance for video content understanding to use the unique attributes of synchronous video reviews for structural modeling.In this paper,we focus on research of structured representation for synchronous video comments.According to the characteristics of the comments,we aim to design the corresponding structured algorithm and to explore the application of video content understanding.Specifically,in this paper,we conduct in-depth research from two aspects:(1)video content topic structured representation learning using danmaku text,exploring how to mine latent danmaku topics and to model gap between danmaku and video storyline.(2)video character structured representation learning using danmaku text,exploring how to extract entity-level sentiment and domain-specific relationship from danmaku text.Driven by the above issues,we focus on the unique language characteristics of danmaku and further explore two perspectives of structured representation learning using danmaku text.Firstly,to model danmaku sequential,we propose a temporal topic structured model and alignment model between danmaku and storyline.Second,to model the language characteristics of danmaku text,we propose a sentiment structured algorithm and structured entity knowledge extraction algorithm.The main contributions of this work are summarized as follows:(1)Topic structural extraction for video-sync comments: Aiming at problem of noise data,strong time dependence,and rapid topic change,we propose a topic extraction algorithm to extract more coherent topics in the time dimension.Experimental results show that the model has significant advantages in the actual dataset.Results show that the model has lower complexity and better topic detection performance for noisy temporal data.To further understand the unique features of danmaku,we analyze data characteristics from the perspectives of time synchronization,emotional,anonymous and video related.This analysis is helpful to facilitate more extensive and in-depth application and analysis of danmaku text.(2)Semantic structural alignment between video-sync comments and storyline: Aiming at the mismatch problem of text styles in video-sync comments and storyline,which is hard to apply previous semantic matching models directly for the alignment,we study a task of aligning video-sync comments(danmaku)to narrative video storylines,and propose to utilize variational auto-encoders to map both user comments and storylines into latent spaces.By posing a matching loss on their latent codes,we reduce their mismatches in the latent space and make the alignment easier to learn.To handle constraints in the alignment,we also apply dynamic programming for finding global optimal outputs.(3)Entity level sentiment structural analysis for video-sync comments: Aiming at the unbalanced distribution problem of entity-level sentiment analysis in danmaku,we propose an entity-level sentiment structural analysis model to predict sentiment for the given entity.To avoid over-fitting and reduce bias,we propose a model with a dynamic masking strategy to learn the representation.The model is based on the pre-training model and shows the effectiveness of this task.We also present a large-scale publicly available corpus for entity-level sentiment analysis in video-sync comments.(4)Knowledge structural extraction for video-sync comments: Aiming at the ambiguous relations define in video-sync comments,we investigate a new task of extract entity and relation on video-sync comments,which can help construct semantic-based video knowledge graphs.On the one hand,we use syntactic rules to extract open relations,On the other hand,we propose a BERT-based model with relationship category constraints to fuse extraction features and to learn the relation automatically.Experiments on large-scale danmaku data have shown the power of our proposed method.