A distributed particle swarm optimization implementation with applications in feed-forward neural networks

This thesis investigates the particle swarm optimization algorithm. Special attention is paid to the use of particle swarm optimization for the training of the weights of feed-forward neural networks. The primary investigations cover distributing the particle swarm optimization algorithm across multiple concurrent processes as well as combining the particle swarm optimization algorithm with backpropagation for the training of the weights of feed-forward neural networks. Three data sets are used for evaluating and training the neural networks. The first data set is the logical exclusive OR operation. The second is simulated grade data for students in a theoretical class. The third is the Wisconsin Diagnostic Breast Cancer data set.;Discussion of particle swarm optimization, distributed computing, and neural networks is followed by a review of research in the area. Experiments focus on the effects of distributing the particle swarm optimization algorithm across two to eight computers. Additional attention is given to the effects of feeding a partially trained network from an iteration-limited backpropagation algorithm into the particle swarm optimization algorithm. Results show that time until convergence for a given error rate is reduced in both the case of distributed computation and the case of combining particle swarm optimization with iteration-limited backpropagation. The only exceptions to this speedup occur for simple data sets where algorithm and communication overhead outweigh the benefits of distributed computing. Furthermore, all data sets began to see a decrease in the amount of improvement after a certain level of distribution.