Research On Image Semantic Acquisition Method Based On Region Of Interest

At present, images and videos are becoming the main forms of multimedia. How to retrieval user needed images effectively from a large-scale image set has become a pressing problem. Conventional CBIR techniques often focus on analyzing the similarity of image visual features to retrieve images. These features often refer to color, shape, texture and the distribution of objects among images. However, the similarity of images is not just depending on their visual features, but more importantly on their semantic similarity. In order to improve the ability of computer vision to satisfy the people's need for understanding of visual information and narrow the "semantic gap" between users and computer vision, we concentrate on the semantic acquisition research based on the extraction of region of interest and apply them to develop a semantic image retrieval prototype system: ISR (Image Semantic Retrieval).The main content of this research include:(1) In view of the existed difficulties of extraction region of interest from general images, we propose a novel method to obtain the region of interest by integrating biological visual attention mechanism. We first extract focus of attention using bottom-up, salience-driven attention model proposed by Itti etc. and then adjust it according to the proposed principles of whole effect and center preference. After that, we obtain region of interest based on the focus of attention and feature saliency map, depending on the characteristics of object spatial proximity and object similarity. Experimental results show that this method can extract different kinds of interested objects effectively under various complex clutters and is highly tolerant to the noise.(2) Integrating visual attention mechanism and object recognition in visual cortex, we propose a novel biologically-motivated computational model for image categorization based on region of interest. We first extract the interested object with the algorithm of extracting the region of interest. Considering the superiority of biological visual system in comparsion with presented computer visual system, we compute a set of position- and scale-invariant C2 features by mimicing the object recognition in visual cortex. Finally we pool them into the standard classifier (Support Vetor Machine and k-Neighbor Nearest) to achieve image categorization. Experimental results show that the average catergorization accurancy can achive as high as 96.65% by using Support Vetor Machine. Compared with the C2 features extracting directly from the whole image, ROI-C2 features perform better on classifing image under various complex clutters, and only just need relatively few training samples.(3) In view of the complexity of images, we adopt multi-instance multi-lable learning to obtain multi-semantic information and propose two novel algorithms: EMDD-SVM and EMDD-KNN to solve the multi-instance and multi-label problems. Firstly, the two algorithms transform the multi-instance multi-label problem to multi-instance and obtain the instance prototypes to represent different kinds of semantic image by using EMDD algorithm. Then they transform the problem to multi-label problem by computing the distances between images and instance prototypes through the minmum weighted Hausdorff distance. After that, the features shift the bag feature space and we make use of MLSVM and MLKNN to label multiple semantics. Finally, EMDD-SVM and EMDD-KNN are applied to categorize images respectively. Experimental results show that the two algorithms can extract multiple semantics from these images effectively and EMDD-SVM algorithm performs better than the MIMLBoost and MIMLSVM algorithms.(4) In order to validate the effectivenss and practicability of our research about semantic acquisition, we develop an image semantic retrieval prototype system: ISR, based on the research results of image semantic acquisition. We discuss the prototype system from the aspects of background, system framework and key techonolgies. Experimental results show that the system can retrieval images by single and multiple semantic keywords effectively and has pretty good practical value.