Dynamic Gesture Recognition Method Based On Address Event Representation Signal

Gesture recognition provides a means of natural and intuitive interaction between human and machine,and has high theoretical significance and practical value.The existing gesture data is mostly taken by a normal camera with frame rate of 30 fps.This frame-based data has a motion blur problem in the case of relatively fast gesture motion,which affects the gesture recognition effect.The non-demand-driven shooting method of normal camera records the complex background environment in the scene,which makes the gesture recognition still facing the difficult recognition problem in complex background.Therefore,event camera inspired by biology and combined with certain specific functions of the human visual system have gradually attracted more attention.The Event Camera is a new type of biomimetic camera based on address event representation(AER)with the advantages of removing redundant information,fast sensing capability,integrated processing,high dynamic range sensitivity,and low power consumption.The demand-driven shooting method of event camera only records the illumination change data caused by the gesture change in the scene.This paper mainly studies a dynamic gesture recognition method based on event camera AER signal.Dynamic gesture data is gotten by using the event camera DAVIS240 C to capture the preset gesture type.Motion information during gesture motion is an important factor in generating AER signal.Therefore,the key gesture in the preset gesture is extracted by using the motion information,and then key gesture is recognized.The main works contain two aspects in the following:1.The gesture is recognized using Convolution Neural Network(CNN).Firstly,the data conversion of three-dimensional AER signal is needed.Then,the converted data is segmented to extract key gestures using Flow Net2.0 which estimates optical flow based on CNN structure.Finally,another CNN network is used to extract the features of key gestures.The experimental results show that the performance of event camera is more accurate than ordinary camera,which further indicates that the event camera weaken the dependence on training data to a certain extent.Compared with ordinary camera,event camera gets the comparable results with less input data.In addition,a dynamic gesture recognition platform is built using the CNN network.2.Aiming to achieve more rational use of the asynchronous characteristics of AER data and more efficient extraction of AER data of key gestures.We use the motion information implicitly brought from the excellent temporal resolution of event camera,select the feature events with important information in the AER event streams,combine these two methods to extract flexibly key gesture event streams.The spatio-temporal features of each feature event are extracted and the Bag-of-Words(BOW)model is used to recognize the preset gestures.The experimental results show that the dynamic gesture recognition based on event achieves accuracy of 97.86%.