Functional Network Theory And Learning Algorithms

Functional network is a recently introduced extension of neural network. There are some basic of theory and application are not perfect, therefore, our research is focused on the problems of novel functional network structure for perfecting basic theories, and a novel learning algorithm of functional network is presented. In this dissertation, we try to use the method of computation mathematics, a new viewpoint is proposed that functional networks can be regarded as a kind of visualization means of some mathematical methods. Then the functional networks representations of mathematical methods or mathematical structure, a novel computation models and methods for traditional numerical computation methods are proposed. Exactly speaking, the functional neuron by way of basic units, base on standpoint of map theory, the complex functional networks are combined with several simple functional networks, this is not just a simple assembly, but according to solving problems of transcendent knowledge. Namely, constitute the topology structure of functional networks. Based on these ideas, the numerical analysis method is adopted to discuss mathtemical essence, including its interpolation mechanism, construct method, approximate theory, and the novel numerical computation method and application fields. It leads to the practical background for there is not intuitionistic background. Main contributions are as follows:1. A functional networks approximation method for multi-dimensional functions is proposed, a kind of separable functional networks for using function approximation is designed, Based on functional networks model, an learning algorithm for function approximation is given, and the learning of the functional network's parameters are carried out by solving linear equations. This can approximate a given continuous function satisfying given precision. The simulation results demonstrate that the approximation method presented in the paper has rapid convergence and powerful performance.2. A characteristic of recurrent functional network (RFN) has forward propagation and feedback propagation is used, the mid-layer of RFN was chosen polynomial function sequence as tunable activation function. A kind of polynomial function recurrent functional network (PFRFN) new model is firstly proposed, which has characteristic of RFN and the capability of function approximate, PFRFN is especially useful for recurrent computation problem. A general criterion on the stabilityof recurrent functional networks is given; the stable points of a network are converted as the fixed points of some function. The PFRFN learning algorithm is also designed, which can perform approximate factorization of multivariate polynomials, this model has much properties such as easily trained and simply structured. However, the numbers of the mid-layer activation function based on this model on the orders of the factorized polynomials. Finally several given examples and learning algorithm show that the proposed model is effective and practical, the learning algorithm is convergent quickly and stable, which can approximate calculate every polynomial factor, The results obtained are very important for study computer algebra.3. A kind of function networks with single input and single output and function network with double inputs and single output as basis functional network model is designed, a new hierarchical functional network is presented. The universal learning algorithm and construction method hierarchical function network is given. Finally, a typical examples of application declared that the hierarchical function networks can be regarded as a kind of visualization means of some mathematical methods, base on hierarchical function network for solve nonlinear algebraic equation system, and researches are made in some technical problems for realizing algorithm. The results show that hierarchical functional network is very suitable for application domains with hierarchical structures. The simulation results demonstrate that the identification method presented in the paper has rapid convergence speed and powerful performance.4. A novel polynomial functional network based on Euclidean of computation model is designed, and a learning algorithm based on Euclidean algorithm is proposed, the learning of parameters of the functional networks is carried out by the solving linear equation. Not only we obtained the exact roots of polynomial equation, but also we obtained the approximate roots of polynomial equation. Finally, the simulation results demonstrate that the identification method presented in the paper, are more efficient and feasible in finding the factor of arbitrary two polynomials.5. The concept of functional network and characteristics of fuzzy interpolation are first introduced, a new functional network modeling, combining constructive characteristics of functional network and fuzzy interpolation mechanism, was proposed, based on which fuzzy functional network is proved to approximate any continuous function defined on closed set to a precision accuracy. This theory result provides us a very useful guideline when we design and application fuzzy functional network, and therefore has very important theoretical and practicalsignificance.6. Functional network is extension of neural network., it deals with general real-valued functional model. In this paper, the structure of functional neuron is changed, and functional neuron is expanded complex-valued neuron. A kind of separable complex-valued functional network model is proposed, a fully complex separable functional network structure that yields a simplified complex-valued back-propagation algorithm is presented. The XOR problem that cannot be solved with two-layered real-valued neural network can be solved by a single Complex-valued functional neuron with the orthogonal decision boundaries, which reveals a potent computational power of complex-valued functional network7. Based on serial functional networks, a learning algorithm of the serial functional networks is proposed. And the learning of parameters of the serial functional networks is carried out by the gradient descent algorithm. Based on this, nine kinds of serial function networks for solving classical functional equations and a kind of solving functional equations method on serial functional networks are presented. The simulation results demonstrate that the identification method presented in the paper has rapid convergence speed and powerful performance. Contrary to traditional numerical method, this method in this paper is used solve general functional equations.8. Functional network is like neural networks, nowadays, there is no system designing method for designing approximation functional networks structure. So, a frame of the constructive functional network design is given, in which the design of the whole functional network breaks down to the design of single neurons one by one. Then, based on GP the design algorithm of single neuron, which realizes the auto-optimization of neuron function types, is proposed. Finally, with many function approximation experiments, it is shown that the proposed constructive functional network design scheme is feasible. Being able to achieve better functional network generalization with small network size.