Scene Recognition Based On Deep Learning

Scene classification is an important issue in the computer vision field It is a challenge task as the presence of intra-class variation, inter-class similarity over the scene images. The great progress on deep learning provides us new solutions for scene classification. Based on the study of traditional scene classification methods and the updated achievement on deep learning, we aim to implement the application of deep learning in scene classification step by step. First, we propose the deep hybrid scene classification network by fusion of traditional scene classification methods and the deep structure. Second , inspired by the class-based saliency of human brain during labeling the scene images, we also present an improved multi-task learning deep network for scene classification.In our paper, we extend the deep hybrid classification structure to local feature extraction layer ,with deep directly connected convolutional autoencoder, following by spatial pyramid fisher vector layer and deep neural networks. Directly connected convolutional autoencoder is a deep directly connected unsupervised structure which reconstructs the input layer from end to end. That makes a difference from the stacked convolutional autoencoder and can get more discriminative mid-features compared with traditional features. In addition, we augment the scene dataset by changing the component of patches selected from origin images. After measuring the similarity between classes, we replace the distinct class-specific patches from the most easily confused class and the corresponding labels will be changed. This can effectively reduce the mis-classification caused by inter-class similarity. We also introduce some patches that are less likely to appear in corresponding scene to generalize across all possible instances of certain categories. The experiments show that our proposed method can improve the classification accuracy effectively.Considering the existing of a large number of labeled scene images in training database,we can invert the class-based saliency maps of scene images. Based on above, a multitask learning method for scene recognition based on class-based saliency is proposed. The algorithm applies selective search method and coarse classifier to generate the class-based saliency images as the reconstruction targets. Selective search is used to generate local areas with the separated elements contained in a certain scene image. Corresponding to their label,then these local images are sent to coarse classifier to get the confidence score of the whole image,which further can be used to generate the class-based saliency scene images. The multi-task deep learning framework for scene recognition can reduce the negative effect of overfitting, the reconstruction network of class-based saliency scene images can effectively works as a supplementary for the classification task, it can resist the interference of non class-related parts of scene images and accelerate the training process of the net. Experiments show that this deep framework can deal with the intra-class variation, inter-class similarity over the scene images and improve scene recognition accuracy.