| Machine learning has made significant progress for many real-world problems. The Deep Learning (DL) models proposed primarily concentrate on object detection, image classification, and image captioning. However, very little work has been shown in DL-based video-content analysis and retrieval. Due to the complex nature of time relevant information in a sequence of video frames, understanding video contents is particularly challenging during video analysis and retrieval. Latent Dirichlet Allocation (LDA) is known as one of the best-proven methods for uncovering hidden latent semantic structures (called the topics) from a large corpus. We want to extend it to capture topics from annotated videos and effectively use them for video classification and retrieval.;This approach aims to classify and retrieve videos based on discovering topics from annotated keyframes in videos. This will be accomplished by employing a pipeline of the following five steps: (1) automatic keyframe detection, (2) video annotation using Show & Tell model, (3) topic discovery using LDA on the annotation, (4) topic assignment to keyframes in the videos, and (5) topic sequence analysis for videos. Mapping the topic histograms of the videos are used to both classify and retrieve videos. The unique contribution of this thesis is to design a topic histogram model that is a new way of representing topics within videos as a sequence and frequency of topics. Based on the framework, we have developed a video application using both Apache Spark and TensorFlow, and then we evaluated different machine learning algorithms and validation techniques using Wikipedia, Flickr30K, and YouTube8M datasets. |
