Multi-level Human Action Prediction Based On Deep Fusion Features

With the progress of artificial intelligence technology and the construction demand of smart city,human-computer interaction application scenarios such as video surveillance and automatic driving C、continuously emerge.Predicting the future behavior of human body can help the machine perceive the behavior intention of human body in advance and improve the effect of human-computer interaction process.In recent years,it has gradually become a research hotspot.This paper studies human behavior prediction technology,which is divided into human behavior posture prediction and uncertainty evaluation of human behavior posture prediction according to different levels.The former aims to improve the usability of human body prediction at the present stage by predicting the specific spatial posture position of human behavior in the future according to the human behavior sequence observed in the past;The latter aims to improve the safety of prediction through offering the reliability of prediction results while giving the prediction results of human behavior posture,so as to guide the safe interaction between human and machine.The main research contents and contributions of this paper are as follows:Firstly,aiming at usability,this paper explores richer deep fusion features of human behavior to improve the quality of human behavior prediction.The existing supervised models ignore the problem of motion coordination.They can model the correlation between local joint point pairs based on human skeleton structure,but they can not model the motion coordination between all joint points,resulting in insufficient ability to extract dynamic depth features of human behavior and distortion of prediction results.Therefore,this paper proposes a coordination attractor(CA)to describe the global motion characteristics,and uses it as a medium to establish a new relative joint point relationship.In this way,all joints can be associated at the same time,so as to explicitly model the overall motion coordination.Based on CA,this paper further proposes a comprehensive human behavior feature extraction framework to model richer dynamic depth features of human behavior.In addition,due to data annotation,the generalization of human behavior deep features modeled by supervised learning is insufficient,resulting in poor prediction effect in complex actions and application scenarios.To solve this problem,a human behavior prediction framework combined with self supervised learning is proposed for the first time.Specifically,this paper proposes an improved version of comparative learning strategy to learn the difference between positive and negative samples and the similarity between positive samples at the same time in semantic space,mine more generalized implicit human behavior depth features,and reduce the data annotation dependency of downstream tasks.In order to improve the representation ability of the model,this paper combines the above two to obtain the deep fusion characteristics of human behavior,so as to further improve the prediction effect.Experimental results on benchmark data sets show that the proposed framework achieves the best performance.Secondly,aiming at safety,this paper studies the uncertainty evaluation of human behavior posture prediction.Previous methods can only provide deterministic prediction values and ignore the evaluation of the reliability of prediction results.In the process of human-computer interaction,wrong or unreliable prediction results may mislead the machine and cause human injury.Therefore,this paper proposes an algorithm scheme to evaluate the reliability of prediction results.Firstly,a predictor of uncertainty perception is designed based on Gaussian model,in which the predicted joint motion coordinates are modeled as Gaussian parameters(i.e.mean and variance).Therefore,the uncertainty of the predicted joint coordinates can be estimated by using the deterministic value.Secondly,this paper puts forward the uncertainty-guided optimization scheme to quantify the uncertainty and help the model converge better.In particular,this paper penalizes the noise samples with high uncertainty in the optimization process to reduce their negative effects.The experimental results show that under the guidance of uncertainty,the model in this paper can be combined with the baseline method to improve the effect,and has great universality.