Fuzzy Linear Analysis Of MLPs And Its Applications

Research on internal behaviors of Artificial Neural Networks (ANN) has been a main subject in its field, especially on the widely used Multilayer Perceptrons (MLPs). These researches have a positive effect upon the further interpretation of internal behaviors of ANN, ANN optimization, performance improvement and knowledge extraction, etc.The classical theory of linear discriminant function is based on classical crisp sets, which is suitable for the analysis of linear classifiers but not for the analysis of classification behavior of neurons and multi-layer perceptrons (MLPs) which activation function are nonlinear, including the smooth-nonlinear activation functions. Theory on simple perceptrons, which can only perform a linear classification, has been consummated systemically. Nevertheless, the more complicated MLPs has not got a systemically theory guidance yet. Based on the classical discriminant function of pattern recognition, along with the fuzzy sets theory, analysis on MLPs used for pattern recognition and classification is carried out. Here the nonlinear activation functions are considered as membership functions. Invoking the concepts of classical discriminant function, new concepts of fuzzy linear discriminant function and fuzzy decision surface are proposed for better understanding classification behaviors of MLPs. Interpretations on MLPs used for pattern recognition and classification are carried out using these new concepts, so as to make it clear and easy to understand the classification behaviors of MLPs. Conclusions named F-LDF is as follows,1) An F-LDF divides the feature space by a fuzzy decision surface or indication surface (hyperplane) net(X) = 0 with a certain fuzzy decision region.2) The normal vector W determines the dimensional orientation of the fuzzy decision surface.3) The location of fuzzy decision surface is determined by the bias parameter b , or the effect of b is merely to shift the decision surface away from the origin.Thus, the MLPs may become fuzzy linear neuron networks.According to the new interpretation of internal behavior of MLPs, a universalmethod of initializing weights is proposed, by which the input-to-hidden weight vectors are initialized and distributed uniformly on a hypersphere in the weight space. This conclusion is as follows,1) The H fuzzy hyperplanes generated by the H hidden neurons divide the feature space with a certain fuzzy decision region.2) The H normal vectors {W;} of H fuzzy-decision hyperplanes respectivelydetermine the dimensional orientations of hyperplanes.3) The locations of//fuzzy hyperplanes will be modified to the optimal locations in learning phase by the modifications of {W7} including {b/}.Simulations on some typical difficult pattern classification problems show that this method can improve the convergence and classification performance dramatically.Medical diagnosis can belong to the category of pattern recognition. Raw data pre-processing, dimension reduction, keeping generalization with accuracy etc, should be solved first when applying MLPs to medical diagnosis. Statistical methods, which are systemic in theory and application, have been widely used in medical field and can be used to utilize dimension reduce of sample data. Meanwhile, method of initializing weights on hypersphere is applied to this application and inspiring results are achieved. The convergence performance of the MLPs is improved greatly. With the increase of sample data, the learning time will increase greatly using weights random initialization method. So the advantage of this novel method becomes important. By using this novel method under the condition of mass sample data, the learning time of MLPs can reduce notably. This can also be helpful to improve the convergence performance of MLPs. This application again justifies the usefulness in improving the performance of MLPs.Enlightened by the thought of Support Vector Machine (SVM), along with the concept of fuzzy linear discriminant function, a novel idea of constructing optimal hypersphere in the feature space for the improvement in the performance of generalization of MLPs is proposed. This novel idea utilizes current MLPs learning algorithm. A method of constructing optimal hypersphere on a small sample set can be achieved using this idea which is shown as follows.Step 1: Given a large parameter a of Sig(-), a rough solution via learningprocess may be obtained which is a hyperplane with narrow fuzzy-region, as isillustrated in Figure 6.2.Step 2: Decreasing a properly, that is increasing the fuzzy region, then input x, one by one and find some samples which subject to eq. (6.3) and/or eq. (6.4).Step 3: Modify {Wy}and {bt} through learning process, that is moving androtating the hyperplane. If the error of classification is close to zero, go back to Step 2; or else it is subject to eq. (6.3) and eq. (6.4), thus the optimal hyperplane is obtained, as illustrated in Figure 6.3.This analysis and method is simple and clear on theory and convenient on applications.