Human Behavior Analysis And Social Relations Recognition Based On Video Deep Learning

Recognizing human behavioral and social relations in video is the important task to understand video semantics.The main difficulty lies in how to analyze and integrate video semantic clues related to human behavior by using depth learning algorithm.In recent years,the traditional deep learning algorithm has achieved good achievements in simple static image recognition,but still can not meet the complex behavior of the video and social relations identification requirements.In this thesis,a semantic recognition algorithm based on Long Short Term Memory(LSTM)model is proposed to identify the behavior of human in video,and then through a node clustering algorithm based on undirected right graph to complete the socialization of human in the video,and finally through the part of the marker factor model SPLP-FGM to infer the human in the video social relations.In addition,this thesie carries out the human behavior semantic recognition experiment on the two data sets of Microsoft video description corpus and film description corpus,and carries on the human social interaction recognition experiment on TV drama Friends data set.The experimental results show that the semantic recognition algorithm based on the LSTM model can effectively identify the behavior semantics of the human in the video,and the partial mark factor graph model can effectively identify the social relations among the human in the video.The work of the thesis is mainly reflected in the following three aspects:(1)Using the convolution neural network to extract the middle semantic features of the human,humans and contexts in each video scene in parallel,the two-level circular neural network is used to fuse the three aspects of semantic information to complete the video The recognition of the human behavior semantics;(2)Mapping the social human in the video into an unqualified image,and completing the social grouping of the humans in the video through a node clustering algorithm based on the undirected right graph;(3)Based on the social grouping and behavior semantic identification of the human in the video,we construct and study the partial mark factor graph model to infer all the unknown social relations in the video.