Algorithm Research On High Level Feature Extraction In Video

In an era of information explosion, it has become an urgent problem incomputer science and technology as how to make computer to automaticallyfind semantics in videos and then index and retrieve them efficiently.Extraction of high level features in videos encounters the same problem ascontent-based image retrieval which is how to bridge the semantic gap. Thecommon way to solve this problem, nowadays, is to extract more effective lowlevel features and at the same time design more powerful classifiers andfusion methods for them. My research then focuses on these three aspects.An effective method to extract shot features is proposed. The unit forvideo retrieval is shot. The common way to extract shot features is firstextracting keyframe from the shot and then computing the image features ofthe keyframe which are then used as shot features. The drawback of thismethod is that it loses most of time information. The proposed Shot temporalwavelet features exploits the wavelet transform of certain kind of imagefeature along the temporal axis to capture corresponding evolving patterns.Experiments show the effectiveness of the shot temporal wavelet features forrepresenting shots with fixed patterns along temporal axis.A novel fusion method for classifiers, AP-based Borda voting method, isproposed. Though it has been studied for a long time, people fail to find acommon effective fusion method for classifiers. The effective fusion methodmay vary according to different applications. The proposed AP-based Bordavoting method uses the negative rank value of each sample as its confidencevalue, and transforms the average precisions on the validate set nonlinearly tothe optimal weights for each classifier. Substantial experiments show theeffectiveness and robustness of our method.A high level feature extraction system, designed and implemented by us,is introduced in this thesis. Most of my efforts focus on the visual subsystem.The visual subsystem extracts many different kinds of keyframe features inseveral ways, then support vector machine (SVM) is trained and used toclassify samples, and finally the results from SVM are fused by our proposedAP-based Borda voting method, which forms the output of our visualsubsystem. The system will further utilize text and time information toimprove the results. We achieved the best average precision in detection of"Basket scored" feature in TRECVID2004.