Research On Dynamic Gesture Recognition Based On Millimeter Wave Radar With The Use Of Deep Learning Methods

Gesture recognition based on millimeter wave radar is a non-contact dynamic gesture recognition method,which uses radar to transmit electromagnetic waves to detect gesture movements.With the advantages of being unaffected by light,good privacy,and easy deployment,gesture recognition based on millimeter-wave radar has great potential for applications in smart home,medical diagnosis,and autonomous driving.Accordingly,it is of significance to study gesture recognition based on millimeter-wave radar for building new human-computer interaction scenarios.It is necessary to introduce deep learning methods into the field to improve the accuracy of millimeter-wave radar-based gesture recognition and further promote the development of human-computer interaction technology.This study focuses on explore the effective feature extraction of gesture features and designs a classification algorithm applicable to gesture recognition.Specifically,three research works in this paper are as follows:(1)Gesture recognition based on multi-residual fusion.There are many shortages existing in current gesture recognition methods such as insufficient feature extraction and insufficient utilization of gesture time series.To address this issue,combining spatiotemporal series,spatiotemporal parallel and spatiotemporal byroad,a multiresidual fusion gesture recognition network was constructed to improve the accuracy of gesture recognition in this study,which used continuous multi-frame range-doppler maps of gestures as the initial input.This method not only make use of extracted singleframe gesture information,but also made use of the temporal relationship between image frames.The results showed that this method achieves higher recognition accuracy with lower parameters compared with other similar methods.(2)Gesture recognition based on dual-stream feature fusion.For addressing the common problem of inadequate use of gesture information existing in current gesture recognition methods,this study introduced I3 D network to extract the effective feature hidden in the distance-Doppler map,and introduced a temporal convolutional network to extract effective feature hidden in distances and angles.And gesture recognition was performed with the effective fusion of the two types of features,which used continuous multi-frame range-doppler maps of and range-angle information as the initial input.The results showed that gesture recognition accuracy based on dual-stream feature fusion was higher than that based on single feature.(3)End-to-end gesture recognition based on 1D-CNN and GRU.To address the issue that the preprocessing of radar gestures in most gesture recognition schemes will bring about the loss of some effective information,this study introduced 1D-CNN to extract feature of radar echo information at a single time point of gesture,and used GRU to extract features of time series of gesture,with the original radar data of gestures as input.This method greatly simplified the process of gesture recognition for retaining the raw radar data without preprocessing.The results showed that without significantly increasing the number of parameters,the gesture recognition accuracy based on this method was comparable with other methods.In conclusion,this study focused on gesture recognition based on millimeter-wave radar,used three different sets of initial gesture features to construct gesture recognition models with deep learning techniques,and the results demonstrated the effectiveness of the proposed models.