| Content-based video annotation is an efficient method for high-performance video index and video search in large video collection and the applications for Internet. This Thesis aims at the construction of a framework for both automatic and semi-automatic video content analysis and video annotation. In this framework, we will discuss the extension and application of machine learning theory for the time-series data (i.e. video data), in which the distribution characteristics of video data is further exploited. Moreover, the statistic learning methods, such as supervised learning, semi-supervised learning and active learning etc., will be applied for improving the performance of the annotation on large-scale video collection, which is important in real applications. As video annotation is highly related with the machine learning and computer vision etc, it is necessary to reconsider these methods from new perspectives, which may further drive the development of machine learning and other visual related fields. The detailed research work is list as follows:Firstly, based on the analysis of the common semi-supervised learning methods, such as self-training, co-training, Co-EM and graph-based semi-supervised learning methods etc., we propose an automatic video annotation framework, in which the semi-supervised learning methods, especially self-training and co-training are further studied for effective annotation on small-size video dataset. The improvements of revisions are on the following aspects. 1. The distribution characteristic of video data is exploited to cluster the video shots in an over-segmentation manner, in which the shots in a certain cluster mostly belong to a same semantic concept. 2. For self-training and co-training methods, the newly added samples are carefully selected, besides that the improvement of accuracy of predictions on these newly selected samples and the model adaptation methods are further studied. Finally, the new self-training and co-training methods, or their derivatives, are proposed which can obtain a satisfactory performance of annotation on certain concept sets and video data collection.Secondly, the active learning schemes are studied. It is known that the active learning process is consisted of sample selection engine and learning engine. In one round of an active learning process, the selection engine selects samples from unlabeled sample pool and requests user to label them before passing to the learning engine. The learning engine then uses a supervised learning algorithm to train or update the classifier with these newly labeled samples. The major limitation of existing active learning algorithms is the efficiency of sample selection criterion, which may not be able to tackle the large variations and complexity of typical semantic concepts in videos. According to these analyses, we further proposed a novel active learning scheme based on multiple complementary predictors, which improves the efficiencies of both of the primary components of active learning. 1) An efficient sample selection scheme is proposed, in which multiple predictors (based on GMM model especially) are applied to find most informative samples. 2) An incremental model adaptation technique, maximum likelihood linear regression (MLLR), is used to update the classifiers which tackle the issue of unbalance between the original training set and the newly labeled data.Furthermore, another novel active learning with clustering tuning scheme, which tries to tackle the disadvantages of current video annotation solutions is proposed based on the margin maximization of the SVM classifier. In this scheme, firstly an initial training set is constructed...
|
PDF Full Text Request