Air-Writing Recognition Based On Deep Adversarial Learning And Domain Transfer

In recent years,with the rapid developments of hardware and the widespread popularity of wearable and mobile devices,the Air-Writing technique has been widely concerned by researchers and the industry.The term Air-Writing refers to a human-computer interaction through writing meaningful isolated characters and strings in the air,and then translating into users' commands towards computers.Compared with the general motion gestures,the Air-Writing provides more abundant expressions.Compared with conventional interaction methods represented by the mouse,the keyboard and the touch screen,Air-Writing is closer to the writing habits of human beings.Among various implementations of Air-Writing,inertial sensor based Air-Writing has broad applications in the fields of smart home and healthcare,due to higher freedom,lower equipment cost and higher environmental noise resistance.However,the weak readability of inertial sensor signals and the scarcity of training data limit the research and development of related algorithms.Based on the above viewpoints,this thesis has carried out research on inertial sensor based Air-Writing,and the major contributions include:1.This thesis proposes two Air-Writing recognition models,one based on Long Short-Term Memory,the other based on Convolutional Networks.The convolutional model uses strip-shaped kernels to adapt sensor data,and uses Fully Convolutional Networks with mean-pooling layer to accept input sequences of any lengths.In the comparison experiments with traditional models,our two deep models achieved better recognition performance on three Air-Writing datasets under different constraints.2.In view of the poor readability of sensor data,this thesis provides a new idea of Air-Writing visualization,and proposes two methods of Air-Writing Domain Transfer,including a supervised Domain Transfer model based on Fully Attention Seq2 Seq,and an unsupervised Domain Transfer model based on Symmetrical Auto Encoding and Latent Adversarial Learning.The two models realize the bidirectional transfer between acceleration and velocity signals,and the spatial trajectories;3.Aiming at the scarcity of Air-Writing data,this thesis proposes a data augmentation method based on Feature Map Position Encoding and Deep Adversarial Learning,which generates samples with controllable quality and certain diversity of categories and lengths.The generated samples improve our classifier's recognition performance by half-supervised training using pseudo-labels.