Research On Matching Relationship Between Video And Music Based On Deep Neural Network

With the development of artificial intelligence technology,many researches in the field of multimedia have made progress.In this paper,by simulating two practical application scenarios,the general concept of "video and music matching relationship" is embodied,and the matching relationship between video and music in different matching situations is explored.Combined with deep learning technology,the corresponding algorithms are proposed.The technical route and work content of this paper are as follows.Aiming at matching scenario 1: Matching suitable background music for short video,this paper proposes a two-branch video-to-music neural network based on cross modal retrieval method.The model projects the content feature data of audio and video to the common subspace through feature selecting,feature mapping and loss function modules.The multi-modal loss function designed in this paper is used to optimize the distribution of data in subspace.Through a series of experiments,along with the subjective and objective evaluation scheme,this paper proves that the matching algorithm in audio and video cross modal retrieval task is outstanding.For matching scenario 2: Recommend music that users may be interested in according to the viewing preferences of video,this paper focuses on the recommender system,and proposes a knowledge enriched attention transfer learning model.The core component of the model is composed of the item feature representation learning module and the user preference mining module.In this paper,the idea of domain adaptation is integrated into the design of item module.Through the strategy of adversarial learning,the sparsity of single domain data is alleviated,and more abundant item feature representation is mined.On the user side,this paper uses the attention mechanism neural network to analyze the user preferences from the user-item interaction data,and then uses the key-value memory network to integrate the external knowledge base data to capture the user's more fine-grained interest preferences and improve the accuracy of modeling.Subsequent experiments also prove the effectiveness of the model.In order to support the work,this paper also makes efforts to construct data sets.The model has achieved good performance in comparison experiments on different data sets,which proves its generalization ability.