Subspace-based semantic concept detection and retrieval for multimedia information systems

The prevalence of digital recording devices, the cheap cost of data storage as well as the convenience provided by the widely accessible Internet have created the demand to retrieve information according to users' requests from multimedia data sources. However, the multimedia information retrieval task has several challenges that need to be addressed, such as bridging the semantic gap, modeling from imbalanced data sets, and utilizing inter-concept relationships to enhance the retrieval performance of an individual concept.;To respond to the challenge of bridging the semantic gap, subspace modeling methods are proposed to address this issue as a classification task. The proposed subspace modeling methods construct a principal component (PC) subspace for each class, where the PCs are derived from the instances belonging to that class. The PCs are selected and ranked based on Fisher's criterion to reduce the searching effort and an iterative searching is utilized to determine the best PC set. Subspace modeling methods are proposed in this dissertation, including multi-class subspace modeling (MSM), binary-class subspace modeling (BSM), and subspace modeling on global and local structures (SMGL). Comparative experiments show that MSM, BSM, and SMGL can outperform some other well-known algorithms on a number of benchmark data sets.;To address the data imbalance challenge, two clustering-based subspace modeling methods called clustering-based subspace modeling (CLU-SUMO) and class selection and clustering-based subspace modeling (CSC-SUMO) are proposed. K-means clustering and/or semantic concept labels are used to partition the majority class (usually the negative class) into several groups, each of which is merged with the minority class (usually the positive class) to form a much more balanced subset of the original data set. Then, the subspace model learned from the original data set is integrated with all the subspace models learned from the balanced subsets to form a classification framework. The experimental results on news and broadcast video data sets support the claim that the proposed CLU-SUMO and CSC-SUMO render better classification performance than some existing techniques that are commonly used to handle the data imbalance problem.;Finally, two ranking strategies that consider the inter-concept relationships are proposed to enhance the retrieval performance from the classification models of a target concept. The co-occurrence class between the target concept and the reference concept is generated and multiple corresponding analysis (MCA) is adopted to capture the correlation between the feature-value pairs (a partition of the attribute values) and one co-occurrence class PP (a class consisting of the instances containing both target concept and reference concept). Such correlation information is used to refine the ranking results from the classification models of the target concept to provide the final ranking scores. The effectiveness of all ranking strategies are attested by the experimental results on public news and broadcast video data sets as well as some image data sets, which demonstrates that the performance of the retrieval results is significantly improved after the proposed ranking strategies are applied.