Research On Algorithms Of Content-based Video Information Intelligence Search

To effectively and accurately search the interested videos for users in the huge amount of video data,the intelligent algorithms of video search based on content are studied.On the one hand,content-based video search method has become a research hotspot.It can realize query and search by means of video index based on semantic concept detection of key frame.This algorithm can build the relationship between the high-level semantics and the low-level features of videos.On the other hand,with the development of computer technology,improvement of artificial intelligence and the large increase of video,the algorithms based on deep learning for video processing attract more and more attention.It can extract spatial features of video and retain temporal information at the same time.The research on video intelligent search method mainly includes shot boundary detection,key frame extraction,semantic concept detection and video search based on deep learning.(1)For the problem of redundancy in video boundary detection,a framework for video boundary detection based on frame entropy analysis and local keypoint features is provided.Segmentation of video into shots by shot boundary detection is first studied for the key frame extraction and video search.It is critical to perform an efficient shot boundary detection algorithm in managing the video information for indexing,browsing,summarization and other content based operations.The proposed algorithm first employs frame entropy analysis to detect the fade and candidate abrupt boundaries and SIFT local features matching between candidate boundaries and their adjacent frames to eliminate the false detection from candidate abrupt boundaries.Efficient shot boundary detection algorithm in turn paves way for the key frame extraction.(2)For the problem when the types of video varied the key frame extraction performance fells due to the selection of single feature,a key frame extraction algorithm based on weighted multi-feature is proposed.A single frame difference measure is first computed as the weighted sum of several frame difference measures corresponding to the individual descriptors for the video shot.The weights reflecting the relevance of each feature descriptor are automatically estimated using a scheme based on the weighed multi-view convex mixture models.Then,these frame difference values are employed to construct a cumulative curve of the frame differences to select the key frames.Those points at the sharpest angles of the curve of the cumulative frame differences are identified as curvature points.Finally,key frames are determined by the frames corresponding to the midpoints between each pair of consecutive high curvature points.This contributes to efficient indexing and search operations by viewing only a few highlighted key frames.(3)To bridge the gap between user semantic and low-level features,provide a algorithm for video indexing mainly based on the semantic concept detection.First,key-points based “bag-of-visual-words”(Bo W)is employed to represent the video images and the representation choices of Bo W feature are optimized.In the semantic concept detection,SVM is employed to build supervised classifiers from labeled data and predicts the labels of other images.Finally,the videos of interest by concept search are ranked and returned to the users.The more the number of key frames searched in the shot,the higher the priority of the shot.(4)For the complexity of the training of deep learning network and the diversity of video actions,a fine-tuning algorithm based on a 3D convolutional neural network model is proposed for video classification and search.The method based on deep learning can directly deal with the spatial-temporal features of video.The 3D convolutional neural network model is first pre-trained on the UCF101 dataset to obtain the initial model.Then the initial model is further trained on the used dataset in the experiment for the fine-tuning of the initialization parameters to obtain the final model.The final model is employed to extract the spatial and temporal features of the test video.Then,Euclidean distance was used to calculate the similarity between video and video in the test dataset.The smaller the value of the distance measure,the higher the ranking priority.Finally,the method returns and ranks the searched video to realize video search.In conclusion,a process of video search algorithm is established based on the underlying features of video images at first.The algorithm has the characteristics of extensibility and robustness for different data types.The algorithm consists of three sections: shot boundary detection,key frame extraction and semantic concept detection.On the other hand,a video search algorithm based on deep learning method is constructed for various video types,and its effectiveness is demonstrated through theoretical analysis and experimental verification.The video search algorithm based on deep learning can use a single neural network model to extract and classify the spatiotemporal features of videos.Simultaneously,it can retain the time information while using the spatial features of videos.So after theoretical analysis and experimental verification,the proposed algorithm can effectively conduct intelligent video information search.