Evaluation Of Saliency Detection Algorithms Based On Practical Applications

Saliency detection algorithms are widely used in many computer vision applications,including object recognition,image retrieval,adaptive compression of image,image segmentation,and so on.In recent years,research works have focused on detecting salient object in natural images or predicting where humans look in the image.There are two problems in the study of saliency detection algorithms.Firstly,most existing saliency detection algorithms assume that the input image is clean and does not have any distortions.However,this situation is not always the case.Secondly,there are many saliency detection evaluation metrics,we usually get different evaluation results from these evaluation metrics for the same salient object detection algorithm.Since the evaluation results are not consistent with each other,it is hard to choose the right saliency detection algorithm for real applications.To solve these problems,we provide an extensive evaluation of saliency detection algorithms in noisy images and a saliency detection evaluation algorithm based on the application of content-based image retrieval(CBIR).We analyze the noise immunity of saliency detection algorithms by evaluating the performances of the algorithms in noisy images with increasing noise scales and by studying the effects of applying different denoising methods before performing saliency detection.We use 10 state-of-the-art saliency detection algorithms and 7 typical image denoising methods on 4 eye fixation datasets and 2 salient object detection datasets.Our experiments show that the performances of saliency detection algorithms decrease with increasing image noise scales in general.We also find that image denoising methods can greatly improve the noise immunity of the algorithms.Our results show that the combination of the nonlinear features(NF)integrated algorithm and Median denoising method works best on eye fixation datasets and the combination of saliency optimization(SO)and color block-matching and 3D filtering(C-BM3D)method works best on salient object detection datasets.The combination of SO and Average denoising method works best for applications wherein time efficiency is a major concern for both types of datasets.We propose a saliency detection evaluation algorithm based on the application of content-based image retrieval(CBIR).The similarity between the saliency maps calculated by a saliency detection algorithm and the ground truth saliency maps is measured by the similarity between the images retrieved by using these two sets of saliency maps as weighting maps.Specifically,we compute the mean square error(MSE)between the ordered lists of the retrieved images.The ascending order of the average MSE over all the images in the dataset is the rank of the saliency detection algorithms in CBIR application.The experiments on MSRA1000 dataset show the top 3 algorithms are SO,SMD,and MR.The experiments on PASCAL 1500 dataset show the top 3 algorithms are DF,SMD,and SO.Among the salient object detection evaluation metrics,WF(2014)is the most consistent with our algorithm.