Research On Optimal Design Of Modular Neural Network And Its Applications

Artificial neural networks(ANNs),inspired by the biological neural systems,are a kind of complex nonlinear system which can process information parallelly.As is well known,the human brain shows prominent advantages in information processing.Its abilities to learn,perceive,and make decisions in complex environments are what the existing neural networks have been trying to achieve.Since the structural connection of the brain determines its functional properties,exploring and simulating the structure features are the preconditions of reaching human intelligence.One essential and substantial feature of cerebral cortex is modularity,which leads to both the basic and important principle of brain—function separation.Consequently,different regions can solve different tasks parallelly.In this study,a brain-like modular design methodology inherited from cognitive neuroscience and neurophysiology is proposed to develop artificial neural networks,aiming to realize the powerful capability of brain—divide and conquer—when tackling complex problems.How to realize the modularized partition structure,how to construct a sub-network with a compact structure,stable performance,and good generalization performance are the key and difficult points in the research field of modular neural network.The research on optimal design of modular neural networks is to obtain effective partition method and sub-network self-organizing algorithm.Firstly,based on the research results of neurophysiology and neuropsychology,we summarize the relationship between the structure connection and function connection of human brain.Then,we apply the spatial modality of brain network into the construct of modular structure.Secondly,after studying the influences on the learning ability and generalization performance of RBF networks,a task-oriented self-organizing design algorithm is proposed to design sub-networks.Finally,a brain-like modular neural network is constructed on the basis of the proposed partition method and constructed RBF networks.The main innovations of this dissertation are as follows:1.Partition methods for modular neural networksConstructing a modular partition structure is the premise to achieve divide-andconquer.Firstly,a partition method based on adaptive resonance theory is proposed due to its good clustering algorithms.By measuring the comparing the similarity of patterns,the whole task is divided to different sub-tasks to construct the modular structure.Then,inspired by the characteristics of brain network and a density-based clustering method,a density-based partitioning method is proposed.In this density-based partition method,the modular structure is obtained by seeking for the hub node.Considering the similarity between RBF neural networks and modular neural networks,the partition method in modular neural networks is transformed to the design of hidden layer in RBF networks.The simulation results show the effectiveness of the proposed partition methods.Moreover,the outperformance of the density-based method demonstrates that simulating the brain structure can improve the partition performance.2.Improved second-order learning algorithm for sub-networksThe learning ability of sub-networks is the guarantee of that of the modular neural networks.In this study,RBF neural networks are applied to construct the sub-networks.Therefore,after analyzing the factors which may affect the convergence performance of the second-order learning algorithms,an improved second-order learning algorithm is proposed for RBF neural networks.The proposed algorithm can improve the convergence speed and convergence performance by transforming the calculation manner of Hessian matrix and seeking suitable initial parameters.Simulation results show that the proposed algorithm can achieve better convergence performance with faster convergence speed.3.A task-oriented self-organizing constructing method for sub-networksThe goal of design or apply ANNs should be achieving the best learning performance and generalization performance with the most compact structure.In this study,a taskoriented self-organizing algorithm is proposed for the construction of radial basis function(RBF)networks(TO-SORBF),in which the structure and parameters are determined simultaneously by the task to be handled.The key aspect of the TO-SORBF network is that it is not designed by human engineers;it is learned from data generated from the task.During the constructing process,first,hidden units are added successively to eliminate the largest instaneous residual error.Then,after the structure growing phase,RBF units with lower significance are pruned to make the entire structure more compact.Both the growing phase and pruning phase are based on the learning results.The quicker the desired learning performance reaches,the less the RBF units are needed.In addition,appropriate initial parameters and activation function can accelerate the convergence process and improve the convergence performance of the utilized second-order training algorithm,which is demonstrated theoretically.Finally,the proposed method is evaluated through a series of benchmark problems,and results comparisons with some state-of-the-art algorithms show its effectiveness and outperformance.Consequently,the sub-networks those are comprised of TO-SORBF network can guarantee the structure complexity and generalization ability of modular neural networks.4.A brain-like design method for modular neural networksA modular design methodology inherited from cognitive neuroscience and neurophysiology is proposed to develop artificial neural networks,aiming to realize the powerful capability of brain—divide-and-conquer—when tackling complex problems.First,a density-based partition method is developed to construct the modular architecture,which can achieve optimal partition structure according to the task to be handled.Then a compact radial basis function(RBF)network with fast learning speed and desirable generalization performance is applied as the sub-network to solve the related problem,which guarantee the parsimonious and generalization of the entire neural network.Finally,the proposed brain-like modular neural network(BLMNN)is evaluated through multiple benchmark numerical experiments,and results demonstrate that the BLMNN is capable of constructing a relative compact architecture during a short learning process with satisfactory generalization performance,showing its effectiveness and outperformance.5.Soft measurement of key effluent parameters in wastewater treatment process using modular neural networksWith the goal to realize the real-time measurement of key water quality parameters in wastewater treatment process,this study constructs a novel soft-measurement model based on the modular neural network.First,based on mutation information and expert knowledge,the easy-to-measure variables which have strong correlations to the effluent water quality parameters are chosen as the model inputs.Then,simulating the modular structure of brain cortex,the effluent water parameters are measured by different submodels,improving both the modeling accuracy and modeling speed.The simulation results based on real data verifies the accuracy and effectiveness of the proposed method.