Research On Visual And Textual Images Retrieval Methods Based On Extracting Salient Visual And Textual Features

The tremendous growth of extensive digital devices has created countless opportunities to capture and share digital images.The excessive amount of digital images has produced huge repositories with the improper organization.Content based image retrieval(CBIR)is considered to be one of the most appropriate practice to index and retrieve large-scale image databases.The core concept of CBIR system is to search and retrieve a number of similar images to the user upon providing a sample image or the other semantic clues such as specifying color,keywords,or sketch.The existing CBIR systems consider the analysis and extraction of visual contents(i.e.color,shape,texture)to retrieve the visual images.However,there is a great need of a system that may consider the textual contents(i.e.text appearing within images)to retrieve visual and textual images.Consequently,this dissertation focuses on new methods for retrieving visual and textual images by considering their visual and textual features.The key concerns of this dissertation are summarized as follows.(1)The problem of searching and retrieving similar textual images by considering their visible text is addressed in this study.This dissertation proposes a novel method that detects the text in textual images and employs that text as keywords to index and retrieve similar textual images.Firstly,the maximally stable extremal region(MSER)algorithm is used that detects the textual regions.Secondly,the two step-filters based on geometric constraints and stroke width transform(SWT)are used to eliminate unwanted false positive textual regions.Next,the remaining textual regions are faded to the next step for optical character recognition.Thirdly,the keywords are formed using a neural probabilistic language model.Finally,the similar textual images are retrieved based on the formed keywords.The experimental results show the dominance of textual features is efficient and effective for retrieving textual images.(2)Since the textual images also contain several fruitful objects and substances that may help to perceive and identify the image.Consequently,this dissertation proposes a method that retrieves similar textual images by considering their visual and textual features.In the first step of visual features extraction,the method extracts and locates the visual salient keypoints.In the second step of textual features extraction,the method detects and recognizes the text appearing within the image.The two feature vectors are specified for both types of features and a Kernel method is used to fuse both the feature vectors.The top rank similar textual images are retrieved based on the fused feature vector.The method allows the user to search the textual images in three possible modes by providing Image query,Keywords,or a combination of both.The experimental results show that combining textual features with visual features can provide effective and efficient retrieval results.(3)Searching and retrieving the textual images is although a good initiative but all the images may not contain textual features rather than they contain only salient visual objects.Both types of contents either visual(i.e.color,shape,texture)or the textual(i.e.text appearing within images)involves fundamental characteristics to perceive an image.Considering this,this dissertation proposes a decisive CBIR approach that is capable to distinguish and retrieve visual and textual images.Firstly,the method classifies the query image as textual or non-textual.If the query image is classified as textual then the text appearing within the image is recognized and formed as Bag of Textual words.Otherwise,the query image is processed for extracting the visual salient features that are formed as Bag of Visual words.Secondly,the visual and textual features are fused together and top rank similar images are retrieved based on the fused feature vector.The method allows the user to inquiry with query image,keywords,and a combination of both.The experimental results show the improved efficiency and accuracy for visual and textual images retrieval.(4)This dissertation proposes a new strategy for searching and retrieving different categories of visual images.In order to improve efficiency and accuracy for traditional search,a new method is proposed that combines low-level visual features with color information.Firstly,the visual salient keypoints are extracted with feature descriptor and quantized into a feature vector.Secondly,the color distribution of the image is extracted and segmented using a non-linear color space model.Thirdly,the similarity for both the feature vectors of visual features and color features is computed.Lastly,the top rank similar images are retrieved based on the resultant vector.The experimental results show the improved efficiency and accuracy of the proposed strategy over state-of-the-art methods.