Study On Models And Applications Of Committee Machines

A Committee machine is an integrated modular system which is composed of a gating network and multiple expert neural networks. The gating network automatically decomposes a global task into several static and temporal subtasks and allocates appropriate expert modules to those subtasks. Each expert learns a subtask to find out its solution, and the solutions to these subtasks are combined to generate a global solution to the global task. It is confirmed in literatures that the generalization of the whole system can be boosted by combining multiple simple expert nets to form a committee machine. But in real-world applications, the task is often too complicated to break down accurately. The complexity of real applications differs widely, which leads to difficulty in tuning the simplicity of the experts in the committee machine. Therefore, to have an accurate task division and subtasks allocation are crucial to realize the potential of a committee machine.Among these problems, focuses are put on the following items.Firstly, the traditional committee machine makes use of fuzzy c-means (FCM) clustering algorithm to decompose the task and the problem is that FCM algorithm can only partition the dataset in which each cluster has similar size. FCM with effectiveness factor (FCMef) algorithm is presented. FCMef algorithm assigns for each cluster an effectiveness factor so as to put the scale of each cluster under control. In this way, FCM algorithm is generalized to FCMef by means of the index of the effectiveness factor.Next, an adaptive FCMef (AFCMef) and a more stable two-phase AFCMef algorithm are proposed. In FCMef algorithm, 1) difference in cluster size is more evident when the index of effectiveness factor takes larger value; 2) if the value of the index of effectiveness factor is too large, some clusters may disappear. However, AFCMef algorithm takes use of above two phenomena to search the optimal index of effectiveness factor heuristically.Besides, experts composed of single-layer linear networks, in which the degree of simplicity is too narrow, are generalized to nonlinear situation, and finally to the more general multi-layer BP networks.Furthermore, general training process of the artificial neural network (expert network) is summarized as well as the influencing factors of the training results. As a result, both positive and negative training frameworks are proposed from the strategic aspect.Last but not least, positive sides of the committee machine and the modifications in this dissertation are confirmed through the experiments performed on three real-world datasets, which are the rainfall, the typhoon tracks on South China Sea and the behavior characteristics of malicious codes. Especially, the regularity of three key parameters is obtained in the parameters modeling for the modified committee machine making use of the rainfall dataset. It is of guiding significance to parameters selection in the similar applications.