The Research And Implementation Of Video Instance Search System On Joint Characters And Scenes

With the rapid development of multimedia and network technology,videos which can hold abundant information has made an explosively growth.Video is becoming an important way to get message and also are changing the habit of working and life styles of everyone.Against this background,how to efficiently retrieve the needed video clips from vast amounts of video information remains a pressing issue in the modern society.Focus its research on TV plays,the paper addressed the video instances search work and was absorbed in joint specific characters and scenes cases.That is,retrieve video clips which contain specific characters in specific scenes from the massive video data.The international authoritative TREC Video Retrieval Evaluation(TRECVID)has described and explored related issues on the subject of joint specific characters and scenes in the year of 2010,that is,instances searching for specific targets in video.Instances search based on the specific targets aims at finding video clips of a certain specific person,object,or place.By contrast,video instances search based on joint specific characters and scenes is intended to retrieve video clips discovering a specific characters in a fixed scene.Obviously,the task of video instances search based on joint specific characters and scenes is facing its most difficult and challenge.Specifically,first,with factors like the shooting angle,shade,light,videos are always presenting volatile content.Especially when the lens transformation between wide angle and particular angle,the content of the scene will happen bigger variation.A single scene retrieval way is difficult to find all the scenes with different content.Second,focus on video retrieval from massive video,video clips which satisfy the query conditions that based on joint specific characters and scenes is few,disturbed video clips are widely existing,for example,the video clips which contain specific characters while have no specific scenes,and vice versa.In addition,the score of characters or scenes in some video clips may be higher,but its adjacent video clips may obtain a low score due to reasons such as block.Third,it is conducive to quickly validate retrieval algorithms if it exists a video retrieval system.And an interface which can intuitively display retrieval results has also become a kind of demand.According to the above problem,this paper launches research from three different aspects,the main achievements were as follows:For the problem that video clips are always presenting volatile content because of reasons such as the shooting angle,shade,light.The paper proposes a scene retrieval method based on combinatorial optimization from both local and global views.On one hand,based on BOW model which using local feature to retrieve the scene marker and then indirectly find the scene.On the other hand,based on CNN model which using global feature to directly find the scene.Finally,fusing the two different retrieval way to obtain the last scene retrieval results.Related experiments are conducted and have shown the effectiveness of our method.For the problem that there exist vast wrong video clips in the massive video which affect the retrieval process,in addition,shots with high score are sometimes existing adjacent right shots with low score which caused by factor such as shade.The paper proposes a video instance retrieval method based on noise removal and shots completion.On one hand,through analysis to find out and then filter those shots which are independent of wanted targets.On the other hand,through Gaussian model to adjust the score of some right but with low score shots which have adjacent shots with high score.The experiment has shown the great progress by this method.For the task of video instance search based on joint specific characters and scenes,this paper accomplish the system of video instance search based on joint specific characters and scenes.Given the gallery and the query instance entity consists of characters and scenes,the system will locate for each query up to the 1000 shots most likely to contain a recognizable instance of the entity.The system has got the best precision in the TRECVID-INS2016.In addition,this paper has designed a display interface with which people can visually catch the results of video instance search based on joint specific characters and scenes.On this basis,this paper have further developed the interactive interface for the system of video instance search based on joint specific characters and scenes.As above,the paper mainly focus on the video instance search technology based on joint specific characters and scenes,and via the issue analysis of video instance search based on joint specific characters and scenes,proposes solutions for corresponding problem.It plays an important role in the improvement of video retrieval accuracy,and further promote achievement transformation and direct application of video retrieval technology in the field of security,commerce,entertainment.