Neural Network Optimization Method Based On Multi Expression Programming And Its Application

Artificial neural networks (ANNs) have the capabilities of learning, parallel processing, auto-organization, auto- adaptation, and fault tolerance. Therefore, ANNs are well suited for a wide variety of engineering applications, such as function approximation, pattern recognition, classification, prediction, control, modeling, etc. However, the performance of ANNs highly depends on the architecture of the networks and their weight parameters. There may be different ANN structure with different performance for a given problem, and therefore it is possible to introduce different ways to define the structure corresponding to the problem. Depending on the problem, it may be appropriate to have more than one hidden-layer, feed-forward or feed-back connections, different activation functions for different units, or in some cases, direct connections between input and output layer. The major problems in designing of ANN for a given problem are how to design a satisfactory ANN architecture and effectively parameters.In recent years, evolutionary algorithms (EAs) have been applied to the ANN's optimization. EAs are powerful search algorithms based on the mechanism of natural selection. Unlike conventional search algorithms, they simultaneously consider many points in the search space so as to increase the chance of global convergence. A novel computational model using Multi Expression Programming (MEP) is researched in this paper, which is encoded by tree-architecture EA, and can optimize the architectures and parameters automatically. The MEP-NN model is proposed in this paper, it can represents and evolves ANN's architectures and connection weights using a linear representation of chromosomes.In the viewpoint of calculation structure, the MEP-NN model can be viewed as a tree-architecture feed-forward neural network with a specific instruction set, which can applies to classification and prediction. Unlike conventional artificial neural network, this framework allows input variables selection, over-layer connections and different activation functions for the various nodes involved. Besides that, it can identify and select important input features automatically to reduce the dimensions.This paper gives a systematically introduce of the artificial neural network, Multi Expression Programming, Evolving Artificial Neural Networks and the Flexible Neural Tree (FNT) from the aspects of basic theory, composition and implementation. Some realization methods and their application, which applied multi expression programming to evolve the flexible neural tree, are proposed in the paper based the former research. The main content is as follow:(1) The paper surveyed artificial neural network and its basic theories. First we summarized the character, generation and development of neural network in details, and emphasized the basic idea, research field and applications. At last we summarized the theoretical and applied research, and sum up the question in designing of neural network.(2) This paper introduced the multi expression programming. First we summarized the basic idea of evolutionary algorithms, emphasized the basic theory of genetic programming, and then introduced the definition and realization of a multi expression programming model and its modified model.(3) The character and realization of evolving artificial neural networks and flexible neural tree were studied in this paper also. A multi expression programming based on tree-structure grammar model is proposed to improve the traditional FNT's disadvantage of optimize the architectures and parameters respectively.(4) The flexible neural tree based on multi expression programming was applied to time-series prediction and classification prediction. The results of simulation experiments, such as the market stock index prediction, foreign exchange rate prediction and gene expression data classification prediction, shown that forecasting results achieved by multi expression programming based flexible neural tree model has better validity and reliability than the canonical artificial neural network model.