Degraded Images Restoration And Classification

Image classification is a very classic task in computer vision,which has important theoretical significance and practical value in image retrieval,medical diagnosis,intelligent security,automatic driving and other fields.In recent years,due to the application of deep Convolutional Neural Network(CNN),the performance of image classification has been greatly improved.But,the current good classification performance is mostly obtained in the clear images.However,in many practical applications,such as autonomous driving,video surveillance,wearable cameras and medical imaging,the resulting images are not always clear,instead,they often contain all sorts of degradations.Therefore,the study of degraded images is an urgent and practical problem.Currently,the image restoration and image classification tasks are mainly studied separately.The existing degraded image restoration methods generally only consider the visual effect,and rarely consider how to serve for the image classification task.Current image classification methods mostly target at the clear images,rather than the degraded images that are more common in real life.This paper takes the degraded image restoration and classification as the main research object,and connects the image restoration and image classification tasks.So that the two tasks promote each other,that is,image restoration task can better serve for image classification and image classification task can guide image restoration,which has very important enlightening significance for the degraded image restoration and classification tasks and helps promote the development of image restoration and image classification fields.In short,this research not only has very important theoretical significance,but also possesses very extensive practical value.The innovation of this paper mainly includes:(1)Aiming at the problem of whether the good classification performance obtained from clear images can be maintained in degraded images,we comprehensively and systematically study the effect of image degradation to CNN-based image classification.More specifically,we first analyze the effect of nine different types of synthetic degraded images and real hazy images collected from the Internet to CNN-based image classification.The experimental results show that the image classification performance drops sharply with the decrease of image quality.Then,we propose methods to improve the classification performance of degraded images.One method is to train and test on degraded images with the same degradation type and level,and the other is to mix degraded images with the same degradation type and different degradation levels to form a mixed training set for training.Although the classification performance has been improved,there is still a certain gap compared with the classification performance of the clear images.Finally,from the feature level,we analyze that the reason for the low classification performance of degraded images may be that the image quality deterioration resulting in the image blur,color distortion and so on.(2)Aiming at the problem of whether the image classification performance can be significantly improved by applying image restoration preprocessing,we comprehensively and systematically study the effect of the image restoration to CNN-based image classification.More specifically,we take image haze removal and image motion-blur removal as examples.Firstly,we use different existing image restoration methods to restore the corresponding types of degraded images;then,we analyze the effect of image restoration to CNN-based image classification;finally,we analyze the correlation between the image restoration and image classification.The research results show that image restoration is not very helpful in improving the performance of subsequent image classification task.The possible reason is that most of the existing image restoration methods only process the input image without introducing new information to the image,and the image restoration process may cause part of the image information to be lost and the image structure is destroyed,etc.,so it is not very helpful for classification performance.(3)The existing degraded image restoration methods often only consider visual effects,and ignore to serve for the subsequent high-level visual tasks,such as image classification,and manual annotations of a large number of category label data is expensive and impractical.Therefore,we propose a image restoration method driven by the self-supervised image classification task.More specifically,we firstly adopt the pixel-level mean square error loss function to train the image restoration network.Secondly,we use the classification of image rotation angles to learn features that benefit identifying the images to further help the image restoration,so that the restored images can help the image classification.To this end,we introduce the image rotation loss function that is utilized to measure the discrepancy between the rotation angle features of the restored image and the clear image,to constrain the image restoration network.Finally,we use the above two loss functions to train the image restoration network to further improve the performance of image restoration.Experimental results in hazy images,low resolution images and motion-blurred images show the effectiveness of this method and effectively solve the shortcomings of existing image restoration methods.(4)Classification models trained on clear images usually contain a wealth of useful information,but the existing degraded image classification methods ignore this useful information,resulting in a large gap in classification performance between degraded images and clear images,which in turn makes the category distribution,feature distribution and visual attention of degraded images are usually inconsistent with that of clear images.Therefore,we propose an end-to-end consistency guided network for the degraded image classification.Concretely,we first propose a category consistency loss to guide the model to learn more in line with the category distribution of clear images.Secondly,we propose semantic consistency loss to guide the model to learn more robust feature representations,making it more consistent with the semantic information of clear images.Finally,we propose visual attention alignment loss,which can align the semantic information regions of clear images and degraded images,thereby improving the classification performance of degraded images.This method is more generalized and suitable for various kinds of degraded images and the experimental results in six degraded images show the effectiveness of this method.