Research On Human Tumble Action Recognition Based On Spiking Neural Network

Action recognition is now widely applied in various fields,e.g.,video surveillance,robotics,game control.Video surveillance can insure the safety of people or environment remotely.When used in home environment,it can identify the tumble action and send alarm in time so as to avoid further danger.In this paper,two kinds of spiking neural networks are used to recognize the human body's tumble action.Firstly,the skeletal movement model of human motion is established.Generally speaking,human body can be regarded as a system composed of rigid skeleton and joint hinge,and human motion can be expressed as the movement of human skeleton.In motion,reliable joint coordinates of human skeleton can be obtained by depth sensor Kinect.Then the X,Y,Z position components of human skeleton nodes are converted to R,G,B components of image pixels,so that human skeleton sequences can be converted into two-dimensional images.Secondly,a spiking neural network based on unsupervised learning is established.Spiking neural networks are more biologically rational than traditional neural networks.The spiking neural network use LIF(leaky-integration-and-fire)neuron model,STDP(spike-timing-dependent plasticity)algorithm,lateral inhibition and intrinsic plasticity.The spiking neural network can be divided into three layers: input layer,processing layer and inhibitory layer.The input image is transmitted to the model at an interval of 350 ms in the form of Poisson distribution pulse sequences.Thirdly,in order to get the accuracy of the above spiking neural network in recognizing human tumble action,a dataset of tumble action is established which includes walking,reading,sitting and tumble action.After transforming human action into corresponding two-dimensional images,experiments on this dataset demonstrate the effectiveness of the unsupervised learning spiking neural network in recognizing tumble action.Finally,establish a convolutional neural network and convert it into a spiking neural network.The dataset used is still the self-built tumble action dataset.After transforming the action in the dataset into two-dimensional pictures of 28 ? 28,a lightweight convolutional neural network is established.According to the principle that the firing rate of the spiking neurons should match the activation grade of analog neurons,the convolutional neural network is directly converted into a spiking neural network.The feasibility of converting convolutional neural network into spiking neural network is verified on the tumble action dataset.