Semantic And Instance Based Video Retrieval

With the advent of information era and the development of internet, millions of video clips are submitted online everyday. It’s becoming an urgent problem to help people find out target videos more accurately, quickly and conveniently through huge dataset.We talked about semantic and instance based video search engines in this paper. The systems focus more on visual features rather than text tags that are quite often used by traditional search engines. Thus we saved much more manual work and time. Our systems support multiple ways of search like video clips, group of images and text based on video content. The main content and contributions of this paper are:1. For different image feature, various strategies of normalization and distance computation are proposed. Experiments show that these improved the search result a lot compared to traditional ways that based on a same strategy;2. For multi-instance search, instance selecting and grouping algorithms are proposed. By color histogram and interesting area comparing, images are grouped and weighted which results in the enhancement of search result. An interactive instance weighting strategy is also proposed to improve the friendliness of the system;3. A search system based on multi-instance is built. And according to different instance type, three re-ranking methods are proposed. For most queries, sensitive image feature are used as complement information to optimize the first-round rank. For queries that related with people, a classifier of face detection is used for image filtering. For queries that related with instance expanding, re-grouping and re-weighting is implemented for re-ranking. Experiments show re-ranking is very needed for result improvement;4. A search system based on semantics is built. Firstly, Automatic Speech Recognition(ASR) result and metadata of the videos are combined for better index building. And then, text search, index and ranking schemes that based on WordNet and improved Lucene is proposed;Our system’s performance is tested by TRECVID2011and2012. We achieved world top two in Instance Search (INS) task of2011and top one of2012as well as world top one in Known-item Search (KIS) task. Our results are much better than the average of all the submitted rounds all around the world. This proved our system’s performance.