Research On Saliency Detection Of Natural Image And Its Application

Recent years have witnessed the rapid increase of multimedia data like image and video.It is highly demanded by people to get use of the large amount of data to build applications that can make people's lives better.Extracting information from images and videos is a basic and important task that can be easily done by human in a very short time.However,computer is still struggling to simulate human's behavior in many tasks such as recognition and classification.Visual saliency model is artificial mathematical system that simulates human's observation behavior by finding the most attractive region in visual stimuli.The most attractive region,also known as salient region refers to where the most important features lie.Most current methods to build visual saliency model rely on integrating bottom-up image features and high-level features.The results are analyzed using visualization techniques such as heat map.Visual saliency is also employed in many other tasks like image understanding and captioning.Still there exist a number of challenges and difficulties.Firstly,only start from the data,the most attractive features remain unknown,as well as the importance weights of different features from different levels,making it difficult to predict salient regions without priori knowledge.Secondly,the acquirement and selection process of saliency database are exhaustive and expensive because of the huge number of stimulus and complicated experimental process.Finally,the correlation between visual saliency and other computer vision domain especially image captioning is important for boosting the development of scene understanding.This dissertation aims to employ new method to improve the natural image saliency model based on the fusion of different levels of features.At the same time,we make further investigation on the saliency data visualization and image captioning.The main contributions are summarized below:1.We propose a new method for detecting and predicting scene saliency.The model is built on the basis of different levels of features including low level features(color,direction and intensity)and scene level features.Additionally,we build a new saliency database especially for scene images and prepare to make it public for further research.2.We propose a new system for saliency data visualization.The system uses Space-Time Cube combined with traditional 2-D visualization method to present a 3-D view for dynamic stimuli,which provides spatiotemporal information of saliency data in an intuitive way,making them easy to observe and understand.3.We explore both low-level and semantic-level features in images,and report observations on the consistency between visual saliency and verbal saliency based on the features.We further demonstrate the effectiveness of our proposed features and predict the consistency between the two modalities on a large dataset with annotations.We evaluate the usability and effectiveness of all our work,making it worthful as references in the visual saliency domain.