Research On Gesture Interaction System Based On Multi-sensor Technology Integration

Human hand is one of the most flexible limbs, and gesture is the body language which is most abundant and expressive. Over the years, computer applications, for instance, strengthen reality and motion sensing games, are developing rapidly, and the traditional human-computer interaction system which is based on keyboards and mouse can not meet the needs of new applications with the limit of fixed, two-dimensional interaction space area. Gesture without these restrictions, with its agile and visualized features, is supposed the best interaction way to virtual environment and natural environment, turning into a new focus of research on human-computer interaction.Owing to a series of uncertainties such as background, illumination and so on, the accuracy of gesture is not ideal. The sensor acquisition scheme is not strict with the background light and other environment, but the sensors are available simple sensor-based gesture solvers exist initial calibration, the still/motion calibration, error and other problems accumulated over time. Therefore, based on detailed analysis of acquisition technology and identification feature of two gestures, this paper came up with an approach based on the combination of information of Kinect depth image and bone, the real-time acceleration of acceleration sensor and real-time angular speed of angular sensor, achieving a high accuracy gesture human-computer interaction system. The main work is as follows:(1) Use the Kinect2.0 to acquire user’s depth image information, then with this information grayscale, smoothing, binarization processing, effectively filter out the background and noise information of depth image. Combine these information and bone information to achieve the predefined gesture recognition with the pre-training results gestures.(2) By analyzing the characteristics of acceleration and angular velocity sensor errors. This paper proposes "stack smooth integration window" preprocessing algorithm of original sampling data. Using this means effectively reduce the noise information of acceleration and angular velocity data and boost the calculation accuracy of movement and posture.(3) For the shortcomings of the acceleration data exists error accumulation when it is used to calculate displacement information, this research put forward a resemblance formula to calculate the amount of displacement data based on Kinect data calculated the amount of displacement to correct the algorithm of acceleration data calculated displacement information. Through the effective integration of Kinect camera and an acceleration sensor, achieve calculating method of the hand movement information in any scenario.(4) For the problems such as initial state can not be calibrated, zero drift and other issues, which exist in posture algorithm process of angular velocity data, this paper raised the use of Kinect data for initial status and motion/static calibration, gravitational acceleration and angular attitude attitude solver in the stationary state solver integration of hand gestures solver algorithms.(5) Designing assembly unit of hand gestures capture which is based on BLE(Bluetooth Low Energy, Bluetooth low energy consumption) 4.0, Acquiring information of space acceleration and angular velocity of hand in real-time to compute the hand motion information and posture.