Research On Multi-modal Fusion Method For Network Video Retrieval

With the development of network and multimedia technology,the number of network video in video sharing websites is exploding.There are three ways of image and video retrieval.Based on the video image retrieval process in the traditional,as the upload provided label,watch user reviews and other information uncertainty,leading to the retrieval results with larger volatility;in the image retrieval process based on video content,quality largely depends on the above video,pictures,coverage rich,images,video,in this process the algorithm based on the time complexity is high,in each link of the data among the adaptive algorithm,the CBIR system is facing great challenges;the traditional multi-mode fusion scheme although to a certain extent to solve such problems,but did not give for web video the specific fusion scheme,and only simple work.The retrieval scheme based on some one or two modes can not meet the needs of "image understanding",or it is not accurate and subject to subjective influence.It is a hotspot to make the fusion of image and video multimodal fusion.Firstly,this paper proposes a novel multimodal fusion network video retrieval scheme.From the visual video content,video title and label text information fusion research and video upload time,category,author three people and produced video interactive social characteristics of heterogeneous information,and this method is applied to large-scale image retrieval tasks in video.Experimental results on Youtube data sets show that:compared with the traditional single feature retrieval scheme and single visual feature,and two modal fusion retrieval scheme,text,visual and user characteristics of our social multimodal fusion specific fusion scheme exhibits better performance.Secondly,in this paper the self-learning algorithm of an active internal parameter tuning algorithm and modal parameters;multi modal fusion retrieval scheme mentioned above two problems,too many modal and modal parameters between internal problems:for the first question,if you do a simple weighted mode in internal,internal mode the weight coefficient is generally only by personal experience are affected greatly by subjective factors.For a variety of social characteristics,through the modal parameters adjustment and automatic iterative optimization,adaptive learning to achieve internal parameters;for the second issue,in the simple application of the one or two modes of the case,the parameter can be selected through the test parameters,the modal characteristics of excessive Shidiao ginseng complex process.This paper examines the classification effectiveness of multimodal structure,using the classification method,modal parameters achieve self-learning purpose.Finally,this paper uses the multi mode fusion scheme to classify the video theme.The experimental results show that the classification of the subject can achieve good results.