Image Content Analysis And Understanding Based On Structured Information

With the booming of social media, we are witnessing an explosion of web images and videos, which therefore turns out a great challenge in multimedia storage, indexing,and retrieval. By mining the internal correlation between the multimedia data, we can better tackle with the big data, and meet this challenge deliberately. In this paper, we will further investigate the structured learning and prediction problems based on the multimedia data. For that, we develop some algorithms from three aspects. The first one is to mine the structured information between high-level semantic cues, which mainly denotes the structure between class and attribute labels of objects. The second one is to mine the structured information between middle-level content of di?erent images. By this way,we can develop more advanced image encryption schemes. The third one is to mine the structured information between low-level features of images, which denotes adding some structured priors into the process of saliency detection and region tagging to improve their performances. Specifically, the research content and contributions contain:In the field of high-level structured information, there are three work. In the first one,we combine the superclass, class, and attributes of objects into a unified hierarchical tree structure. Furthermore, a prediction model based on structured support vector machine is proposed to accomplish the structured prediction output problem for an unknown object.The proposed method augments the image description, and helps bridge the semantic gap between users’ intension and the low-level image features. On this basis, the second work incorporates the spatial information into the multiple tree-structured semantic units,and performs the structured image retrieval. In the third work, we present a high-order descriptor based on the class, spatial and scale cues of objects. This descriptor integrates multiple context cues and high-order relations into an unified framework. Experiments show its e?ectiveness.In the field of middle-level structured information, we mine the correlation between multiple di?erent images. We proposed an image reconstruction method based on the coupled dictionary learning and compressive sensing. This method has two steps: encoding phase and decoding phase. In the encoding phase, multiple images are encoded by enforcing the same sparse coe?cients in coupled dictionary learning. In the decoding phase, given one of the encoded images. the other images can be reconstructed. This method is then used to solve the image encryption problems. Compared with the traditional methods, the proposed method does not need to embed data into the cover image,which greatly improves the security of secret image in the transmitting process.In the filed of low-level structured information, there are also three work. The first work uses the tensor to describe facial images, and presents a robust face clustering method based on tensor decomposition. The proposed method preserves the spatial information of faces and is more robust to the noises within facial images. In the second work, we mine the tree structure between feature points, regions, and images. Moreover,we integrate this tree structure into the framework of structured sparsity, which leads to an improvement of performance of region tagging. The third work is to mine the structured information between images when considering the saliency detection. The structured priors are incorporated into the D-S evidence theory to accomplish the whole task. The proposed method is simple and e?ective. There is no training phase used, which can be regarded as a post-processing step for the existing saliency detection methods.Among the four problems of image content analysis and understanding, the first problem contains two algorithms: the structured saliency detection algorithm and the high-order contextual descriptor. The second problem contains the robust face clustering algorithm. The third problem also contain two algorithms: the structured region tagging algorithm and the augmenting semantic prediction algorithm. The final problem contains the structured image retrieval algorithm and the meaningful image encryption algorithm.