Training artificial neural networks using graphics processing units

Artificial Neural Networks (ANNs) are computer-based models that try to simulate the structure and/or the functional behavior of neurons and process information using the connectionist approach to computation. Training ANNs using the back-propagation algorithm is a very powerful machine learning system. However, as these networks become larger and more complex, the computational effort required grows significantly. Graphics processing units (GPU) are cost effective, highly accessible devices specifically designed to exploit parallel floating point operations applied not only to computer graphics, but also to scientific computations. The high-parallelism inherent to the GPUs makes these devices especially well-suited to address ANNs problem. The goal of this thesis is to find efficient GPU implementation for training ANNs using back-propagation algorithm. The CUDA framework is used for the GPU implementation. We implemented ANNs algorithms on three applications: handwritten digit recognition, soil type recognition through satellite images and gas analysis system. We evaluated our algorithms to compare the speed between two different implementations; one sequential and one parallel. These algorithms produce impressive improvements, up to 10 to 40 times faster, depending on different ANN architectures.