Research On WiFi Signals Based Fine-grained Gesture Sensing

In view of the vigorous development of the new generation of information technology such as mobile network and Big Data,the machine centric computing mode is changing towards the human centered computing mode,which is also a process from human adapting to machines to machines adapting to human,"human-oriented" intelligent human-computer interaction has gradually become a new trend in the development of computing mode.Gesture sensing is an important enabler of intelligent human-computer interaction.Compared with existing sensing technologies such as radar and computer vision,gesture sensing based on WiFi signals has the advantages of low cost,easy deployment,non-infringement of privacy,and freedom from the influence of light and occlusion,which has become an emerging research hotspot in this field.However,due to a series of endogenous defects such as low sensing resolution and clock asynchronization of transceivers,the state-of-the-arts are mainly focused on the method of pattern recognition,which faces the challenges in practical scenarios:the sensing models are heavily dependent on the training environment,with poor generalization abilities to unfamiliar environments.Therefore,with the goal of improving the generalization ability,this dissertation conducts research on the WiFi signals based transferable gesture recognition model,and environment-independency gesture tracking model,aiming to achieve ubiquitous,robust,low-cost and high-precision gesture sensing.The main contributions are summarized as follows:1.Research on transferability oriented contactless gesture recognitionIn order to improve the generalization and transferability of gesture recognition model,this dissertation introduces a transferability oriented contactless gesture recognition scheme.In particular,aiming at the huge cost of manually collecting training samples,we first design a Conditional Generative Adversarial Networks(CGAN)based data augmentation mechanism,and apply twoperson zero-sum game theory to combat training and generate virtual samples with similar data distribution to expand the dataset.Then,we propose a Maximum Mean Discrepancy(MMD)based feature transfer method to measure the domain discrepancy of different environments in the reproducing kernel Hilbert space,and leverage a multi-kernal MMD and multi-layer adaptation scheme to improve the transfer efficiency.The experimental results demonstrate that this scheme can improve the accuracy of 71.2%in inexperienced environments,realizing one training and simple adaptation.2.Research on three-dimension oriented contactless gesture trackingIn order to fundamentally solve the problem of insufficient generalization ability,this dissertation introduces a three-dimension oriented contactless gesture tracking scheme.Specifically,we first construct a three-dimensional dynamic tracking model based on the geometric analysis of Fresnel Zone model and Angle of Arrival(AoA).Secondly,we propose a super-resolution multiple signal classification(MUSIC)algorithm,which combines the phase information between antennas and subcarriers to expand the virtual array unit,and thus improve the sensing resolution of spatial multipath.We then design a probabilistic joint modeling based on spatial search and MUSIC pseudospectrum to accurately estimate the initial positions of gestures.Finally,we propose a interference signal elimination algorithm based on peak detection and linear interpolation to remove the signal interference components which are unrelated to gesture movements.The experimental results show that confronting different environments and user conditions,the scheme can achieve a median tracking error of 2.5cm in three-dimensional space.3.Research on NLoS scenario oriented contactless gesture trackingIt is difficult to accurately measure the prior positions of the transceiver in the Non-Line-of-Sight(NLoS)scenario,and the gesture initial position cannot be estimated as well.In such case,it is infeasible to construct a gesture tracking model,which is the well-known position-dependency problem in the field of gesture tracking.In order to deal with position-dependency problem,this dissertation introduces a NLoS scenario oriented contactless gesture tracking scheme.To be specific,we first construct an incremental displacement model to shift the sensinng observation from the transceiver based global view to the antenna array oriented local view.Unlike the traditional global view,which focuses on the spatial absolute position of the gesture at each moment,this scheme focuses on the gesture relative motion trace in successive times,thus eliminating the dependence on the prior position of the transceiver and the initial position of the gesture.At the same time,considering the stability of wireless links,the weighted least squares method is used to optimally integrate the estimated results of each link.Finally,we propose an automatic gesture segmentation algorithm and a continuous Wiener process acceleration model based Kalman filtering algorithm to further improve the tracking accuracy.The experimental results attest that this scheme can achieve a median tracking error of 0.9 cm without knowing the prior positions of the transceiver and the gesture initial positions.4.Research on remote sensing oriented hand-held mobile device trackingThe contactless sensing paradigm uses the hand-reflected signal for gesture sensing,which has the advantages of non-intrusiveness and user convenience.However,the reflected signal power is usually weak,and the effective sensing range is limited to 2m.In order to realize sensing whenever and wherever that user wants in the whole space,this dissertation introduces a remote sensing oriented hand-held mobile device tracking.Limited by the WiFi sensing resolution,traditional mobile device tracking methods such as triangulation can only achieve decimeter-level tracking accuracy,which is far from meeting the fine-grained gesture sensing requirements.Unlike traditional methods(e.g.,triangulation),this scheme introduces the time-domain correlation characteristics of target motion by constructing an extended incremental displacement model,and extends the contactless incremental displacement model to the tracking scene of hand-held mobile devices,which improves the sensing accuracy while solving the position-dependency problem.Then,we design a novel reference array to not only eliminate a series of frequency offsets caused by the asynchronous clock of the transceiver,but also specify the positive direction of the gesture.Finally,we formally integrate a resistance-type grasp pressure sensor into the mobile device to determine the validity of gesture movements and segment successive gesture traces.The experimental results show that this scheme can achieve a median tracking error of 0.85cm in 10×10m2 indoor environments without the prior positions of the transceiver.In summary,this dissertation studies WiFi signals based gesture sensing.Aiming at various challenges in practical applications,a series of gesture sensing schemes are designed,from gesture recognition to gesture tracking,from contactless to hand-held mobile device.The effectiveness of each proposed scheme is verified through theoretical analysis and practical experiments,laying an important theoretical and technical foundation for the practical application of WiFi signals based gesture sensing.