The Research On Human Behavior Recognition Based On Multi-Kinect In Human-machine Cooperation

With the rapid development of science and technology,more and more robots are used in the field of vision scope.But it can't completely take the place of people to deal with complex tasks,because the person have the ability to analyze,make decisions and deal with complex environment and tasks.If we want robots to complete complex tasks in the complex working environment,it is necessary to discuss the cooperation between human and robot in the operation environment.Human-computer cooperation refers to the ability that robots can understand human behavior through voice,gesture and body movements,so as to ensure the safety of the human body in the working environment.At present,it is a difficult problem in the human-machine cooperation.In this paper,computer vision technology is applied to human-machine cooperation,and multi-kinect is used to recognize human behavior in the working environment.Skeletal tracking technology of K inect can be able to real-time track human body joints,however,when we use a single K inect,sometimes it exists obstacles,and human body often appears self-occlusion while they are moving,these will result in the data of K inect errors or even lost,which will make human behavior recognition is not accurate.Therefore,this paper uses two Kinects to monitor the human body at the same time to solve the inaccuracy of human behavior recognition caused by occlusion.In this paper,first of all,we use the human body as the target,then we use the skeleton tracking technology of K inect respectively to obtain the 3D coordinates of 25 body joints in two K inects;Secondly,we transform the joints coordinates which are captured by two Kinects in different views to a common world coordinates by using coordinates transform,then we fuse coordinates of two identical joint nodes in two K inects to obtain a complete human skeleton model,as a res ult,we can get the convinced human pose.We use the human skeleton model to extract human body area to set up the bounding box of human body model,so that we can prepare for subsequent human-machine collision in advance;Finally,we can use the human skeleton data to extract the vector of human behavior feature and sent them to the neural network to identify human behavior.After the computer identify the human behavior,we can convey them to the robot in the real-time,then the robot can make corresponding action according to the human behavior,this not only can make the human-machine efficient cooperation,but also it can ensure the security o f the people in the working environment.The thesis mainly completes the following work:(1)Research the color image?depth image and skeleton tracking technology that are acquired by the K inect,then we acquire the relative position of the two Kinects in human-machine cooperation.(2)Based on the three-plane target reconstruction technique,the K inect external parameters were calibrated,then,the 3d coordinate data of human joints collected by two K inects were unified into the commam coordinate system.Firstly,the cloud information of three targets is collected.Then the random consistency algorithm is used to obtain the same name vector and the same name point in two Kinects.Finally,we use the spin theory to make the coordinate data unify the comman coordinate system.(3)This paper puts forward the data fusion algorithm based on reliability.We use human physiological movement constraint to establish a hierarchical skeletal model and a unified global skeletal model,then we use them to calculate the human joint euler Angle.We use the unified global skeletal model and human joint euler Angle range to judge whether the joint data is reliable,then we fuse the trust data.If it has missing data after fusion,we will predict the data by using kalman filter algorithm.(4)The human body area is extracted from the unified global skeletal model after fusion,and the bounding box of human body model is established so as to construct the dangerous space in the subsequent human-machine collision detection.(5)Use the K inect to extract the vector of human behavior feature,then we send them to the classifier to learn and identify human behavior.