Research On Scene Understanding In Computer Vision Based On Topic Models

Scene understanding is one of the ultimate goals in computer vision research. Scene understanding is a high-level vision task, which includes how to recognize ob-jects in sophisticated scenes, how to discover interaction among objects, or how to figure out when and where the events take place. Scene understanding requires in-tegrate many other research in computer vision, for example, combination of textual and categorical information based on object recognition and segmentation for scene understanding tasks. Recently, topic models, which are based on the bag of words assumption, have been widely used in object segmentation and achieved satisfactory results, however, they also has some drawbacks such as the unreasonable assumption that pixel patches and topics are generated independently, lacking the modeling of the relationship among entities in a scene, and the insufficient use of information from other modalities.In this paper, we propose a unified probabilistic graphical model named Topic-based Coherent Regions Annotation (TCRA) model for automatic image regions anno-tation. Our model extends LDA by imposing neighborhood constraints on topic level through a Markov parameter, and incorporating an annotation part for learning and in-ferring labels. We also provide mean field variational inference for model learning. Our model provides the following two advantages. First, spatial information is mod-eled explicitly and therefore we will obtain more coherent result of regions annotation. Second, only image-level labels are needed instead of associating them to particular pixels or regions in the training stage, and the association between labels and image re-gions can be inferred, effectively alleviating human labeling burden. As a result, given an image without any textual prior, its regions could be automatically labeled based on the learned model. Experiments conducted on two datasets illustrate the performance of our method.