Research On Optimization Of Multi-cost Sensitive Backpropagation Neural Network

The learning of Backpropagation Neural Network (BPNN) aimed at lowering the classification error, usually assuming that all the samples had equal price when misclassifications were made. Constructed based on such an assumption, the BPNN might lead to pay heavy prices when used as a classifier even if the misclassifications took place rarely. For this reason, minimizing the total misclassification error of a sample set has become the current research focus on BPNN and decision-making support system (DMSS) constructing.The current cost sensitive BPNN learning usually takes into account one type of cost in adjusting the distribution of the samples, relabeling of the classes of the samples, or modifying the error evaluation function into cost sensitive. The major disadvantages of the current learning methods are: (1) one-type cost assumption is not agreed with the actual case where there usually exist several types of costs; (2) minimizing the total misclassification price can not or hardly give attention to the classification accuracy at same time.In order to pay attention to both the classification accuracy and the misclassification price, a BPNN learning method that does not need adjusting the distribution of the samples, relabeling the classes of the samples, and modifying the error evaluation function is presented. This learning method is based on a genetic algorithm, which takes the classification error and several types of misclassification costs as optimization objectives, looking for the best BPNN optimal at all objectives by applying the Pareto multiple-objective optimization theory.To prevent the genetic algorithm from falling into a local optimum, the strategy of sharing fitness among all individuals in a nich is adopted. To automatically ascertain the radius or the range of the nich, a new method of discovering the individuals' even-distribution inflexion is presented based on the psychological judgement of whether a distribution is even. And based on this method, a new genetic algorithm which can automatically search the radius for niches is developed.As complex information system usually contains large amount of attributes which can be used as the input variables of a model, it may bring about constructing a complicated BPNN. To construct a structure simplified BPNN, an enumerative and a heuristic method of attribute reduction are put forward respectively. The enumerative method prunes the super sets of the reduced attribute sets bottom up, based on a circular queue used for enumerating, and thus it remarkably enhance the enumeration speed of all reduced attribute sets. The heuristic method emploies a genetic algorithm in reducing the redundant attributes of all attributes. Besides taking fitness function as its heuristic function for its selection operation, the genetic algorithm uses the attribute's information entropy to construct the heuristic function for its mutate operation, and in this way it markedly elevate the convergence speed of attribute reduction.The parallel implementation methods for the attribute reduction algorithm and the multicost sensitive BPNN learning algorithm have been studied respectively, and the multicost sensitive DMSS has been developed based on these algorithms by using of the object oriented technology. To validate the performances of the previous mentioned algorithms, the datasets from the public UCI and actual clinical database collected by the Chinese National 863 Target-oriented Project "Grid Based Digitalized Medical Treatment Decision Making Support System" are used for experiment.The experiments show that the bottom up parallel attribute reduction algorithm is able of dealing with large scale attributes, and compared with the single-cost sensitive BPNN constructed by changing the error evaluation function, the multicost sensitive BPNN has higher classification accuracy and lower misclassification price when inputed with equitable costs.