Multi-modal Abnormal Gait Recognition Based On Generative Adversarial Network

With the aggravation of the aging of the population and the increase of disabled people,more and more attentions have been paid to the secondary injuries caused by rehabilitation training and falls.Timely and accurate identification of abnormal gait in daily life and rehabilitation training is the key to prevent users from falling.Compared with the wearable sensor method and the video method to recognize abnormal gaits,the size and quality of the dataset have a great impact on improving the recognition accuracy.The generative adversarial network has strong samples generation ability to enrich the dataset and improve the recognition accuracy.When the robot assists in rehabilitation walking,the recognition method based on the combination of far and near distance has higher accuracy.Firstly,a method of recognizing normal and abnormal gaits by a wearable attitude sensor is studied.Firstly,the wearable attitude sensor module is the combination of gyroscope and magnetometer.Secondly,the Kalman filter is used to filter the measured acceleration,attitude angle and magnetic force.The cross product algorithm and Mahony algorithm are used to calculated and corrected the attitude.Finally,the attitude sequences are classified by one-dimensional convolution neural network.Secondly,the abnormal gait images are extracted and the gait images recognition input pipeline is constructed.First of all,the Gaussian mixed model background subtraction algorithm is used to extract the moving targets as the dataset.Secondly,a moving target tracking method based on particle filter algorithm is proposed to determine the moving target position in real time.Finally,the combination of moving target tracking and moving target extraction is used as the input pipeline of gait images recognition.Then,the generative adversarial network augments the gait sequences and images dataset.First of all,the principle of generative adversarial network(GAN)and the Auxiliary classifier GAN(ACGAN)structure of generating gait images according to species and the Information maximizing GAN(Info GAN)structure of generating gait images according to posture are analyzed.Secondly,combined with one-dimensional convolution neural network,the gait sequences generative adversarial network is proposed to generate gait sequences.Then,according to the species of gait images and the combine of deep convolution neural network,ACGAN and Info GAN propose abnormal gait generative adversarial network(AGR-GAN).Abnormal gait is generated according to species and posture,and the generalization ability of the network is verified by MNIST dataset.The abnormal gaits recognition network is obtained by transfer learned discriminator of AGR-GAN.Finally,the gait images are augmented by AGR-GAN to verify the accuracy of the proposed method and model.Finally,the combination of abnormal gaits recognition and intention recognition is applied to the walking robot.First of all,the acceleration collected by the attitude sensor is filtered out the gravity component and the horizontal attitude angle is obtained,which is used as the input of intention recognition together with the forearm pressure.Secondly,the linear kernel and the support vector machine algorithm with the slack variable is introduced to identify the user's intention.Finally,the effectiveness of the method combining intention and abnormal gait recognition is verified.