Algorithm Research And Hardware Design Of 3D CNN For Action Recognition

Nowadays deep learning technology develops prosperously,which is applied to many fields including human action recognition task.Three-dimensional convolutional neural networks(3D CNN)are the mainstream deep learning method to tackle human action recognition which acquire good results in academic research,but there are also some difficulties to solve before its large-scale application.The largest difficulty is that the effective networks is too computationally intensive to deploy in mobile device.To address above difficulty,this thesis researches the algorithm of three-dimensional convolutional neural networks and design its hardware accelerator.The goal of algorithm research in this thesis is to reduce data bit width and decrease calculation amount in three-dimensional convolutional neural networks.In order to achieve the above objective,this thesis improve the previous conventional binarization method to a binarization method friendly to hardware design.Besides,existing lightweight methods has been considered to improve three-dimensional convolutional neural networks in this thesis.Combine the above two methods,this thesis proposes a model named lightweight binarized three-dimensional convolutional neural networks which acquire 89.2% accuracy in KTH human action dataset proposed by KTH Royal Institute of Technology.In addition to algorithm research,a dedicated hardware accelerator is designed in this thesis.In the design of accelerator,a method for convolution acceleration and a storage mode for feature map in on-chip memory are presented.The accelerator with above design method has the advantages of low off-chip memory bandwidth demand and reduced on-chip memory usage.Eventually the accelerator is implemented by FPGA,and it achieves the performance of 380 FPS frame processing speed and 64.37GOP/s/W energy efficiency.Though based on small-scale FPGA platform,the accelerator in this thesis can also attain high frame processing speed and excellent energy efficiency which fully satisfy the real-time requirements of real recognition application.It is hoped that this thesis' s accelerator can be used to build practical human action recognition system at mobile device in the future.