Prior-knowledge Based Neural Network Modeling And Application

Accurate models are important to the research and application of chemical engineering. However, most problems in chemical engineering are complex and we know little about their principles. So it is difficult to build accurate models directly by the first principles. On the other hand, although neural networks can build models without the principles, it completely replies on sample data. When sample data are not enough, or contain noises or aberrant data, they cannot reflect the properties of the object to be modeled very well. Thus the model's performances are degraded. Prior-knowledge based neural network models can overcome the disadvantages of the above two methods. It takes parts of the principles from the object as the prior-knowledge, and combines them with the sample data to build a reliable model. In this paper, we mainly discuss the design, implement and application of the prior-knowledge based neural networks, including the following parts:1. According to the different ways in which the prior-knowledge is encoded into the network, the prior-knowledge based methods can be divided into three classes: namely Architecture Constraint (AC) method, Weight Constraint (WC) method and Data Constraint (DC) method. AC method selects the architecture of the network (including the network structure, the fashion of the computation, etc.) that satisfies the prior knowledge. WC method modifies the training algorithms in such a way that the training trajectory in weight space converges to a point in a weight subspace that satisfies the prior knowledge. DC method encodes the prior knowledge in the network by preprocessing the training data to reflect the prior knowledge better.2. A kind of AC method - exponential weight (EW) method is proposed. It replaces the weights in the network with exponential weights to perform the training and simulation. Thus the network will conform to the prior-knowledge of increasing monotonic in nature. And it will also not violate the prior knowledge after training.3. A kind of WC method - constrained optimization (CO) method is proposed. It takes the network training process as to solve a nonlinear optimization problem. The network weights are optimization variables and prior-knowledge is the constraint. The optimization problem can be solved use algorithms in the optimize theory.4. A kind of WC method - adaptive (AP) method is proposed. From the viewpoint of system theory, it takes the network as a system and the training process as the self-evolvement of the system. When the network satisfies the prior knowledge, it is in stable status; contrarily, it is unstable. Initially, the system is stable. In the evolvement process, the system may possibly enter into an unstable status. At that time, the AP method will adjust the system parameters to induce it back to the original stable status or enter into a new stable status. By continuously evolving and adapting, the system finally reaches the training goal.5. A kind of WC method - improved differential evolution (IDEP) algorithm is proposed. IDEP algorithm also takes the network training process as an optimization problem. Differing from CO method, IDEP algorithm is based on the differential evolution algorithm. It finds the optimum by applying selection, crossover and mutation operations on a group of solutions iteratively. The flip operation, LMD strategy and RP strategy are the keyparts of the IDEP algorithm, which make the solutions conform to the prior-knowledge, speed up the evolution process and prevent from premature respectively.6. A kind of DC method - interpolation method is proposed. In this method, some interpolation data which satisfying the prior knowledge are inserted into the original training data, then the compound data are used to train the network. In training process, the added artificial data are deleted step by step while the training goal increases gradually until it reaches a predefined value.7. An idea of designing hybrid methods is propos...