Learning Prototype With Deep Architectures: An Approach To Classification

One essential task of AI is to model our world, extract semantic information andrepresent the world. Such a task can be extracting semantic concepts which humanbeings can understand from a picture constructed by pixels. This is a complex task andneeds to abstract or make non-linear embedding on difierent layers. Embedding resultsof one layer are used in higher layer, thus these layers of embedding construct a deepnetwork. Deep networks are such learning structures that automaticly extract featureson multiple layers. One typical deep structure is deep (multiple layers) artificial neuralnetwork. For classification, a deep artificial neural network extracts features layer bylayer from bottom to top and classify at the highest layer. Weights of network areupdated by back-propagation algorithm to achieve good performance.Resent years, researchers have proposed a training method called unsupervisedpre-training to help train a deep network to reach better performance. Essentially, theunsupervised pre-training is learning to pre-extract features. However, the real worlddata may contain complex structures and noises, features extracted by unsupervisedmethods cannot represent data properties appropriately. Usually, most tasks we wantcomputers to learn have strong purposes, such as classification. Therefore one needsto direct a computer to learn meaningful features. On the other hand, results leaned bymachines should be understandable and more than a class label.In this thesis, we propose learning prototype in deep architectures and apply thismethod in classification. We introduce prototype autoencoders as basic blocks in deepnetworks. In a prototype autoencoder, noisy data are given as input and to fit corre-sponding prototype through hidden layer. By using this structure, autoencoders aredirected to learn useful features from noisy data and reconstruct prototype by these features. To further make use of expression ability of deep networks, we stack proto-type autoencoders to learn feature expression on difierent levels to construct a deepprototype net. We conduct experiments on noisy MNIST and rectangle datasets tocompare the classification performance with other deep networks. Experimental re-sults show that deep networks based on prototype autoencoders have good ability offeature extraction and achieve high precision of classification.