Vision-based Analysis Of Person-to-person Interaction

Person-to-Person Interaction analysis and recognition is a hot topic in the domain of computer vision and pattern recognition, and has promising applications to intelligent surveillance, visual reality and motion analysis. Key problems in this task are feature extraction, feature representation and action recognition. This thesis focuses on Person-to-Person interaction analysis and recognition from image sequences.A hierarchical approach for recognizing person-to-person interaction in indoor scenario from a single view is proposed based on spatial-temporal feature extraction and representation. The dense space-time interest points detected from videos are divided into two sets exclusively according to the history information along the evolvement and the connectivity of the two -person silhouettes. Then K-means clustering performs on points in the training set and learns the spatial-temporal codebook. For a given set of interest points, a spatial-temporal word is built by allowing each point to vote softly into the few centers nearest to it and accumulating the scores of all the points. The Conditional Random Field (CRF) whose inputs are the spatial-temporal words is used to modeling the primitive actions for each person, and common sense domain knowledge and first order logic production rules with weights are employed to learn the structure and the parameters of Markov Logic Network (MLN). The MLN can naturally integrate common sense reasoning with uncertain analysis, which is capable to deal with the uncertainty produced by CRF.Experiment results on our two-person interaction dataset are provided to demonstrate the effectiveness and the robustness.