Research And Application Of Flexible Neural Tree Based On Grammar Guided Genetic Programming

Research indicates that artificial neural network as a powerful method has been applied to a number of science and engineering fields, such as approximation of function, system identification, signal and image processing, time-series forecasting, and so on., for their massively parallel processing, good fault-tolerant, auto-organization, auto-adapted ability and the association function.Among the research of artificial neural networks, the most important problems are how to select the appropriate architecture and relative parameters for an artificial neural network. Weights and biases of artificial neural networks can be learned by many methods, i.e. back-propagation algorithm, genetic algorithm, evolutionary programming, random search algorithm and so on. Usually, a neural network's performance is highly dependent on its structure. The interaction allowed between the various nodes of the network is specified using the structure only. There may be different ANN structure with different performance for a given problem, and therefore it is possible to introduce different ways to define the structure corresponding to the problem. Depending on the problem, it may be appropriate to have more than one hidden-layer, feed-forward or feed-back connections, different activation functions for different units, or in some cases, direct connections between input and output layer. Recent years, there are many researchers devoted themselves to optimizing the architectures and parameters of neural networks automatically. Flexible neural tree is a kind of artificial neural network, which is encoded by tree-architecture, and can optimize the architectures and parameters automatically. For the tree-architecture based coding, some tree-architecture based optimization algorithms, such as genetic programming, ant programming and probabilistic incremental program, and etc., can be used to evolve the architectures of flexible neural trees.This paper gives a systematically introduce of the artificial neural network, grammar guided genetic programming and the flexible neural tree from the aspects of basic theory, composition and implementation. Some realization methods, which applied grammar guided genetic programming to evolve the flexible neural tree, were proposed in the paper based the former research. The main content is as follow,(1) The paper surveyed artificial neural network and its basic theories. First we summarized the character, generation and development of neural network in details, and emphasized the basic idea, research field and applications. At last we summarized the theoretical and applied research, and sum up the question in designing of neural network.(2) This paper introduced the grammar guided genetic programming. First we summarized the basic idea of evolutionary algorithms, emphasized the basic theory of genetic algorithm and genetic programming, and then introduced the definition and realization of a grammar guided genetic programming based on context-free grammar model and one based on stochastic grammar model.(3) The character and realization of flexible neural tree were studied in this paper also. A grammar guided genetic programming based on context-free grammar model and one based on stochastic grammar model, which were combined with parameter optimization algorithms, such as genetic algorithm and particle swarm optimization algorithm, were applied respectively to establish the flexible neural tree models.(4) The flexible neural tree based on grammar guided genetic programming was applied to time-series prediction and classification prediction. The results of simulation experiments, such as the market stock index prediction, foreign exchange rate prediction and leukemia cell classification prediction, shown that forecasting results achieved by grammar guided genetic programming based flexible neural tree model has better validity and reliability than the canonical artificial neural network model.