Research Of Adding Visual Saliency To Image Retrieval Framework

With the development of Internet,multimedia information on the Internet is increasing.As an important element of multimedia information,images are closely related to people's life.Thousands of data being uploaded and downloaded every day on the Internet.In the vast image database,how to quickly and accurately query the image information that users want to search is crucial.Content based image retrieval extracts the key information points of the images and conducts similarity measure to retrieve the similarity images.The content based image retrieval method is faster and more accurate than the text based image retrieval method.However,this content based image retrieval method needs to fully excavate the content information of the images to find the real similar images,which has a high requirement for image content understanding.Visual saliency information as a kind of human attention can excavate the important information of images.This paper mainly adds the visual saliency information to image retrieval framework.The main research work including:(1)The CNN architecture research of embedding training visual saliency.This paper proposes a CNNr architecture of embedding training visual saliency.The architecture combines DeepFixNet model with VGG model and adds the new convolution layer after the eighth layer of DeepFixNet model to train the saliency map which applied to image retrieval task.The experiments on the six classical data to verify the validity of saliency information.The generalization experiments use the proposed model to verify the validity on fuzzy datasets.The experiments find the advantage of visual saliency on fuzzy datasets.The visual saliency can help the image retrieval find the similarity of fuzzy images.(2)The research of adding visual saliency to CNNs.This paper proposes a two-stream CNNs framework MAC model.The MAC model consists of two branches.The main branch is the VGG model.And the auxiliary branch is the DeepFixNet model.The two branches fuse the feature maps in convolutional layers and add the visual saliency information to VGG model.The experiments on the four classical data sets finally verify the validity and extensibility of the MAC model.The retrieval results improve 4 percentage on MAC model.In the experiment,the MAC model is found to fully excavate the image detail information and improve the accuracy of retrieval.(3)The application of visual saliency in complex image retrieval scenes.Adding the rain/snow noise to four kinds of classic data sets,simulating a real complex environment background.Using the MAC model for image retrieval and image classification tasks,preliminary study advantages of visual saliency in complex environment on image retrieval task.The retrieval results improve 8 percentage in complex environment.