Saliency Based Image Segmentation And Scene Understanding

As one of the ultimate goals of computer vision, scene understanding has always been investigated by many researchers in past years. Technically speaking, the task of scene understanding is to let the computer know what contents are included in images. To achieve this tremendous task, many subtasks have contributed a lot to this goal, such as image segmentation and object recognition. Recently with the rapid development of electronic devices and internet, not the number of pictures increase in billions year by year but the contents in these pictures become to exhibit more and more variations. This phenomena naturally lead to a more general concept of scene understanding. That's to how to let the computer know the themes for so many pictures. And this task is so urgent that several leading labs and companies are developing their own information retrieval system. On the other side, scientific research on visual saliency and scene understanding has proved some exciting improvement. Several exciting computational models have been proposed by researchers and they have demonstrated efficient effects.Motivated by advancement promoted by those labs and researchers we propose to develop a system that incorporate the cutting-edge research and can help computer to learn more about our real world. We start from several subtasks that have been investigated for many years, they are saliency detection, image segmentation, image annotation and image classification. It has been a long time since saliency detection, image segmentation, image annotation and image classification are investigated separately. But recently experiments on biology sheds a light that animals including mammals and other animals have eyes understand the contents of scene with the help of selective visual saliency and visual memory. This indicates that image saliency, visual memory ,the understanding of scenes and the association of meaningful things are correlated. Inspired by this mechanism we propose to construct a computational model to perform the task of understanding and association of scenes. First we construct the models of visual saliency and image segmentation similar to the mechanism of selective saliency and the organization of visual information. The two models are used to filter and extract the meaningful object in the scene. Then combined with the association of visual information we annotate the key meaningful objects using information we get from images similar to the to-be annotated image. At last we perform the understanding and association of meaning unit and concept theme using probabilistic graph model.The difficulty of our research is how to let the computer know, from the perspective of human beings, the meaning of images and recognize the difference between different types of scene and the similarity within the same scene. Through modeling saliency of meaning objects, key meaningful units annotation and the precise modeling of scene types, this paper made an efficient attempt to minimize the gap between bottom-up features and the top meanings in images.