Research On Methods For Home Video Content Analysis

With rapid adoption of consumer digital recorders as home appliances to capture people's memorable experiences and document their daily lives, the quantity of home video data is increasing dramatically. However, home videos are usually characterized by the low visual and audio quality, as well as non-edited content. In order to alleviate the average home users' effort to browse, retrieve and share personal videos, content analysis has become an important research issue in home videos for turning the raw data into a well-organized and easy-to-access database. Home video content analysis aims at not only providing personalized multimedia services for home users, but also enhancing the development of related issues in video content retrieval, such as semantic modeling, machine learning and information retrieval. Therefore, the research on methods for home video content analysis has both important theoretical and applied values.This thesis investigates the key problems of home video content analysis at three different levels: quality assessment, content understanding and content representation. Quality assessment is a fundamental step for filtering home video content due to the low visual quality. Content understanding enables modeling of semantic concepts at a higher level than quality. Content representation corresponds to the highest application level which can provide a compact and efficient representation of video content based on quality and understanding of content. As a result, these three levels build up a relatively comprehensive framework for home video content analysis. Accordingly, this thesis conducts a deep research on home video content analysis, and obtains the following achievements:(1) For quality assessment, we propose a novel spatio-temporal quality assessment scheme for home videos, in which a set of key spatio-temporal factors as well as the relationship betwen these factors and the overall visual quality are investigated. In contrast to existing frame-level-based quality assessment schemes, a type of temporal segment of video, sub-shot, is selected as the basic unit for quality assessment. A set of spatio-temporal visual artifacts, regarded as the key factors affecting the overall perceived quality (i.e. unstableness and jerkiness as temporal factors; infidelity, blurring, brightness and orientation as spatial factors), are mined from each sub-shot based on particular characteristics of home videos. The relationship between the overall quality metric and these factors are exploited by three different methods, including User Study-based, Rule-based and Learning-based. To filter home video content, we present a scalable quality-based home video summarization...