| Tiny machine learning(Tiny ML)aims to bring machine learning to extremely edge devices,i.e.,microcontrollers in close proximity to sensors,promising to unlock entirely new smart applications and pave the way for the last mile of artificial intelligence.Meanwhile,gesture recognition,as an important technology for realizing human-computer interaction,is of great significance for realizing convenient interaction between people and smart devices.Currently,the more mature static gesture recognition solutions include the use of data gloves or vision cameras for static gesture acquisition,which have many limitations in practical applications,such as cumbersome wearing process,affected by lighting conditions,etc.,and the processing platform mostly relies on cloud or edge servers.In this thesis,focusing on Tiny ML applications,a static hand gesture recognition system based on an ultra-low resolution infrared array sensor and a low-cost AI chip is designed and implemented on the microcontroller side to further reduce privacy risks as well as costs.On the one hand,it contributes to the continuous progress of Tiny ML;on the other hand,it helps to promote the comprehensive application of gesture recognition technology in real life.The main research of this thesis includes the following aspects.(1)A method for rapidly collecting and labeling sensor data in Tiny ML applications is introduced.The structured data is generated in real time by a microcontroller-driven sensor and forwarded to a third-party online platform for online collection and annotation of sensor data.(2)A pre-processing method for temperature array data on static gestures is investigated.Through interval mapping and data reconstruction,the one-dimensional temperature array data is transformed into two-dimensional pseudo-image data,and the data set about static gestures required for model training and testing is constructed.(3)The neural network structure is highly customized for the limitations of the used microcontroller itself.Based on the idea of lightweight design of the model,an ultra-lightweight neural network model is designed with deep separable convolution as the core building block,and the operators in the model structure are adapted to the hardware accelerator equipped on the microcontroller to achieve full acceleration of the model inference at the hardware end.Before model training,various data augmentation strategies are adopted to achieve the augmentation of the data set,which in turn enhances the generalization of the model.(4)Specific implementation of the static hand gesture recognition system on the microcontroller side.A neural network compiler is used to convert the model format for the deployment of the neural network model on the microcontroller side.Pre-processing of the temperature array data generated by the ultra-low resolution infrared array sensor was re-implemented on the microcontroller side,including performing interval mapping and upsampling by a bilinear interpolation algorithm for the input of the neural network model.Finally,the heat map of the static gesture and the inference result of the neural network for this static hand gesture are simultaneously output to an LCD screen for display.The experimental results show that the static hand gesture recognition system based on an ultra-low resolution infrared array sensor and a low-cost AI chip designed and implemented in this thesis has an accuracy of 99.14% for several simple static hand gestures,and the model inference time at the microcontroller side is about 35 ms,which can achieve accurate recognition in real time,and is low cost,anti-interference capability and good privacy. |
