Two-dimensional Gesture Interaction Design Based On The Combination Of Software And Hardware In PYNQ

Traditional human-computer interaction information input methods have certain limitations.With the development of technology,the application of gesture recognition provides a new solution for human-computer interaction.As the second language of human beings,gestures can directly convey individual information and are more flexible and convenient in interaction.Computer vision-based two-dimensional gesture interaction technology is becoming more and more mature.In order to significantly reduce the interaction delay,higher-performance computer equipment is required,and the use cost is relatively high and limited to computer use.In response to this problem,the paper proposes a two-dimensional gesture interaction design solution based on Xilinx's PYNQ development platform.The main content of the thesis is to realize a human-computer interaction method with gesture action as the core.The camera collects two-dimensional gesture action images,and recognizes and locates gestures in the image based on features such as skin color and shape,analyzes continuous gesture combinations and positioning information in real time,and realizes control operations of translation,rotation,and zoom.Finally,the gesture control command is output in the form of a serial port frame.The paper chooses PYNQ as the development platform for two-dimensional gesture interaction design,and uses the design idea of combining software and hardware to realize the overall function of the system.The design of each hardware module in the gesture interaction design is realized by PYNQ's FPGA part,and the configuration of each hardware function module and the process control of the overall system function are realized by the software system.This solution can make full use of the FPGA's high-speed parallel computing advantages and the flexible control capabilities of the software system to meet the requirements of gesture interaction design to achieve high-speed image processing.The paper mainly completed the following tasks:(1)In the FPGA part of PYNQ,the hardware function of gesture interaction design is realized,the Verilog hardware description language is used to realize the image collection and the design of gesture interaction output function module,and the HLS high-level synthesis tool is used to realize image preprocessing and gestures.Design of extraction and gesture recognition function modules.Finally,the hardware function module is mounted on the PYNQ software system bus to form a complete gesture interaction design hardware system.(2)A Linux-based software operating system was built on PYNQ.In the software system part,the driver program of each hardware function module and the writing of gesture interaction design program were completed to realize the configuration of hardware function modules and the overall flow of gesture interaction.control.(3)A gesture interaction test environment was built to test the gesture interaction design;the test results show that the paper basically realizes the analysis of translation,zoom and rotation operations and the output of control instructions,and the average operation delay is within 55 ms,which basically satisfies the gestures Design requirements for low-latency interaction.By comparing the time it takes for software and hardware to implement the same function,it highlights the speed advantage of image processing in hardware implementation.