The Design Of Sign Language Recognition System Based On Multi-sensor And Zynq

Gesture refers to the hand movements made by the individual according to subjective consciousness.As a body language,it contains rich information and is an efficient and natural communication method widely used in daily life.As a kind of gesture,sign language gesture is an important way for deaf and mute to communicate and communicate.However,deaf-mute people have difficulty communicating with normal people who do not understand hand Gesture,which also reduces the enthusiasm of deaf-mute people to communicate with the others.This paper designs a hand Gesture recognition system based on multi-sensor and Zynq in which the other person does not need to hold a camera,the system has good real-time performance and can rapidly recognize 15 frequently used hand gesture in sign language.The system design can be divided into two main parts: sensor data gloves and Zynq hand gesture recognition system.The sensor data glove uses the Arduino with ATmega328 P as the hardware platform,which is responsible for controlling the collection,processing and transmission of sensor data.The flex sensor and the inertial measurement unit are used to obtain the five-finger bending degree information and the hand posture information respectively,and the data of the two sensors are used for hand Gesture recognition.At the same time,the CRC-16/CCITT-FALSE verification process is added at the data transmitting and receiving ends to solve the problem of data loss of the Bluetooth module during data transmission.BP neural network is used as a hand Gesture recognition algorithm,and the algorithm is designed and simulated in MATLAB.At the same time,in order to hardwareize BP neural network efficiently and quickly,this paper uses High-Level Synthesis technology to implement BP neural network for identification on the Programmable Logic of Xilinx Zynq SoC,multiple optimization schemes are compared from the perspective of resources and timing,and a scheme that achieves a good balance between resources and timing is obtained.The actual results show that the algorithm design and implementation can be completed at a higher level of abstraction using High-Level Synthesis technology.BP neural network can be realized efficiently in FPGA.Compared with traditional RTL design,the design cycle is greatly shortened,and can achieve a good performance.The design of Zynq identification system uses soft and hard collaborative design.The Processor System is software part,which is responsible for the data processing tasks executed sequentially,receives the data sent by the sensor data gloves through the Bluetooth module,and then retains the complete data through the CRC check process,extracts and normalizes the sensor data,then works with programming logic to complete the OLED display.Programmable logic as hardware part improves the real-time performance of the system,it is responsible for not only the implementation of the neural network which is computationally intensive and can be processed in parallel,but also the SPI interface protocol which was inefficiently executed by the Processor System.The Processor System cooperates with Programmable Logic by calling the neural network module and OLED acceleration module through AXI4-Lite bus,and completes the gesture recognition and result display of hand Gesture.Experiments show that the system's recognition of 15 hand gesture basically reaches the level of theoretical accuracy.