Embedded Implementation And Algorithm Optimization Of Gesture Recognition Based On Convolutional Neural Network

With the rapid development of information technology and the maturity of computer vision technology,vision-based gesture recognition applications have gradually become an important research direction in Artificial Intelligence(AI)field.As a very natural method of semantic expression,gesture plays an important role in Human Computer Interaction(HCI).However,the complex gesture structure and changing environmental factors increase the difficulty of gesture recognition and lead to low recognition accuracy.The Convolutional neural network(CNN)has improved the problem that traditional identification methods need manual selection of features.It has been widely used in video surveillance,HCI and big data analysis.The real-time requirements for algorithm implementation in these fields are also increasing,which makes it urgent to study how to use FPGA to accelerate the CNN algorithm.Therefore,this paper combines the gesture recognition technology to study the embedded implementation of convolutional neural network in ZC706 platform.The specific works are as follows:(1)For the problem of complex gestures structure,lighting,background and environmental factors will affect the recognition accuracy and artificial selection of characteristics is difficult to adapt to the gestures variability,this paper presents a gesture recognition scheme that combines skin color models and CNN.For different background gesture images,we choose the appropriate skin color model to segment the gesture area,and then the gesture region is extracted and reconstructed using algorithms such as morphological operations,mean filtering,and connected component labeling algorithm.Finally,this paper combines the deep learning CNN method and constructs a network model based on gesture grayscale images.The experimental results show that CNN can perform feature learning efficiently.The average recognition accuracy of gestures in the self-built dataset and Massey dataset reaches more than 98%.(2)Compared with the PC implementation of the gesture recognition system,the embedded implementation has the advantages of more flexibility and wider application.Taking into account the real-time requirements of the gesture recognition system,and the calculation mode of CNN is very suitable for hardware acceleration,this paper presents a FPGA-based CNN accelerator scheme.Xilinx's Vivado HLS development environment can convert the synthesizable high-level programming language C/C++ into an RTL-level implementation to shorten the development cycle.Therefore,this paper adopts pipeline optimization,loop unrolling,memory optimization and fixed-point quantization to implement a 7-layer CNN accelerator for gesture recognition in the Vivado HLS based on the ZC706 platform.Experimental results show that our CNN accelerator runs at 200 MHz,which achieves the peak performance of 22.04 GMACS and the power efficiency with a value 16.76 GOP/s/W on ZC706.It is 126× faster than Core i5 2450 M CPU and 10× fasterthan NVidia GTX 840 M GPU implementation and achieves 12× power efficient compared with GPU.(3)For the problem of poor real-time performance of gesture recognition algorithms running on embedded platforms,combined with the architecture of ZC706 platform FPGA+ARM,this paper adopts hardware and software cooperation to design and implement a prototype system.The system implements video capture and gesture segmentation on the ARM side,and hardware acceleration of the gesture recognition algorithm based on CNN,image transmission,and HDMI high-definition display are implemented on the FPGA side.It explores a convenient,low-cost,low-power embedded solution for gesture recognition.