Research On Deep Learning Algorithm With Visual Impression

Visual impression is the visual information stored in memories and is an important form in the process of visual cognition. People achieve sensory organ stimulation via vision and form the images of cognitive objects by the information process of brains. Visual impression is the basis of human brains to complete various complex tasks efficiently. For the most of time, brains directly process the visual information with the help of visual impression in memories, rather than with blind computations. When people is recognizing cognitive objects, since eighty percent information source derived from vision and people always use existing visual impression to cognize present objects in the tasks such as object recognition, research on visual impression plays an important role in pattern recognition and computer vision.Deep learning contributes deep nonlinear networks by stacking single-layer module and can implement infinite approximation of complex functions. Deep learning can gradually learn different abstract features in an unsupervised way and supply an expressive distributed presentation. Because of the characteristic of auto-learning, deep learning reduce the manual design cost of features relatively to the traditional feature selection methods. The efficient feature extraction way of deep learning brings break-through results in application of many areas, such as object recognition.Human visual system has a deep architecture. From the simulation of biological mechanisms point of view, this can become a strong proof supporting deep learning. To some extent, deep learning simulates the layer-wise process of human cognition. The deep structure of brains determines that visual impression abstract the visual information step by step. This means that the mechanism of visual impression can correspond to the deep learning techniques. Thus, the combination of visual impression and deep learning is a potential research direction. How to simulate visual impression in human cognitive process by deep learning methods becomes a problem that needs urgent solutions. This problem requires the designed deep learning algorithms on one hand can reflect characteristics of visual impression, and on the other hand can complete relative visual impression functions. On the basis of deep learning, how to extract hierarchical visual impression efficiently and how to guarantee the robustness to small variations are both the main problems in the designing of deep learning algorithms.The deep problem of visual impression is one of the key problems of deep learning. This paper aims at hierarchical feature and stable feature of visual impression and achieves the following achievements.First, we develop two visual impression models: recognition model and generalization model to simulate the cognition process of human visual systems. We show how the visual impression learned with a deep neural network can be efficiently transferred to other visual recognition tasks. By reusing the hidden layers trained in an unsupervised way, we show that we can largely reduce the number of annotated image samples in the target tasks. Experiments show that parameters estimated in the source task can indeed help the network to improve results for object classification in the target tasks.Second, we present in this thesis a novel approach for training a topological deep neural network with visual impression. We show that by combing denoising auto-encoder model and contractive auto-encoder with Hessian regularization model, we can achieve a deterministic auto-encoder aiming for robustness to small variations of the input. We exploit the tangent propagation algorithm to show how our algorithm can capture the manifold structure of the visual impression and build a topological atlas of charts. Finally, we show that by using the learned features to initialize a deep network, we achieve superior classification with relatively smaller parameters than some other models.Third, we exploit a novel algorithm for capturing the Lie group manifold structure of the visual impression. By developing the single-layer Lie group model, we show how the representation learning algorithm can be stacked to yield a deep architecture. In addition, we design a Lie group based gradient descent algorithm to solve the learning problem of network weights. We show that our proposed technique yields representations that significantly better suited for training deep network and is also computationally efficient.In conclusion, the innovation points of this paper include(1) new methods of Lie group visual impression representations for deep learning,(2) measurement methods of visual impression for deep learning, such as hierarchical measurement, topological measurement and Lie group measurement,(3) new deep learning algorithms with visual impression.