Research On Some Key Techniques Of Content Based Video Retrieval

On account of the dramatic development of computer, digital multimedia and Internet techniques, a sea scale video resources are available. Effectively retrieving interesting video clips from abundant video databases have been considered a very important and challenging research topic. Content based video retrieval techniques have emerged for meeting need of customers. In this dissertation, the exploratory research work has been done regarding video retrieval, which includes video feature extraction, shot boundary detection, key frame extraction, shot similarly measure and clip retrieval and so on. The main contributions of this dissertation are summarized as follows:(1) A video feature extraction approach based on improved locality preserving projections is proposed. In order to solve the problems relate to the projection dimension and nearest neighbor K in locality preserving projections (LPP), we get the optimal projection dimension based on structure error between dimension reduction before and after, then we dynamically select nearest neighbor K based on the neighbor statistical character of each data. On accont of not overcoming defect, which signal feature fails to present the image content, we bulited a new feature, mixing color, texture, and shape, and then the feature of new video reduces to a lower one though the improved locality preserving projections.(2) A novel method to detect video shot boundary based on localized multiple kernel SVM (L-MKSVM) is introduced. This method consists in first building intermediate features integrating local temporal-spatial information, which are input to an efficient supervised classifier to identify shot boundaries, and then video frames are split into boundary frames and non-boundary frames with L-MKSVM. In order to improve the detection precision of MKSVM based on global optimization, we project video frames to hashing space with locality sensitive hashing algorithm, and then construct L-MKSVM for each hashing subspace with multiple kernel learning methodology. Meanwhile, we create a balanced data set about video boundary frames and non-boundary frames with SMOTE over-sampling technique.(3) A novel approach based on the multivariate time series segmentation is proposed for key frame extracting. We present an improved multivariate time series segmentation algorithm (IMTSSA), which defines segmentation cost function thorough between-series scatter and within-series scatter. This algorithm can avoid calculating complex cost function and number of segmentation for traditional multivariable series segmentation algorithm. In order to gain the optimal point of segment, we present a series segmentation strategy of combining separation and combination. On this basis, video shots are divided into some sub-shots with similarly content through IMTSSA, and then the frame closest to the sub-shot centre is chosen as the key frame.(4) A new method of shot similarity measure based on shot pattern similarity and visual similarity is reported. In order to solve problem, which content is similar but large features difference, we present conception about shot pattern similarity combining video feature and shot change model. We calculate the pattern similarity by dynamic time warping technology (DTW), which speeds the process of pattern similarity adapting Keogh function. Considering the significant factor of key frame, we present a proportional weight Kuhn-Munkres algorithm (PWKM). We calculate the visual similarity combining PWKM and longest common subsequence algorithm (LCSS). Final, we calculate the shot similarity by weighting pattern similarity and visual similarity.(5) A video clip retrieval method based on clip is proposed. Considering the inaccurate of sliding window segmenting video clip, we segment similar video clip from video with FASTA algorithm. Employing maximum mathching Hungarian algorithm, is to delete the pseudo-similar video clips. Final, we calculate the clips similarity by weighting visual similarity, temporal order, granularity and interference of shots.Finally, the dissertation is finished by asserting some open issues on content-based video retrieval, and pointing out some further research directions.