Study On Human Behavior Understanding In Videos Based On Deep Metric Learning

Due to the increasing demands for video content analysis,artificial intelligence has attracted more and more attention by researchers.For public safety and security,it is required to understand human behaviors.The human behavior understanding technology can be applied to video surveillance,human-computer interaction,and multimedia semantic indexing.Human behavior understanding is a complex,diverse,and challenging task that has been widely concerned in the previous work.Motion detection,motion estimation,motion tracking,scene modeling,and behavior understanding are basic techniques for human behavior understanding,which have been actively studied in recent years.With the rapid development of computer vision and machine learning,video content analysis has achieved a remarkable advance in performance.Surveillance cameras are widely used in public safety and crime prevention,and intelligent video surveillance needs abnormal behavior recognition to identify events that do not conform to normal behaviors.Modeling human behavior and activity patterns to identify or detect special events is a key issue for human behavior understanding in recent years.An intelligent vision system is established to understand the video content and perform semantic reasoning based on the observed dynamics of the moving target in video.For video surveillance,video content retrieval,and human-computer interaction.Human action recognition is a basic task.It recognizes human actions based on the observation of human behaviors in a video and generates a label.In this thesis,we mainly investigate human action recognition and abnormal behavior recognition in videos based on deep metric learning.The research scope of this thesis is as follows:1.We propose deep metric learning for abnormal behavior recognition based on multisimilarity loss.For the accurate recognition of abnormal behaviors,it is required to analyze and learn the difference between the abnormal behaviors by distinguishing abnormal video from normal video,i.e.anomaly detection.Moreover,the key to the anomaly detection lies in learning discriminative features.Discriminative feature representation enables our model to obtain good recognition performance.Specifically,we adopt deep metric learning and multi-similarity loss to learn the difference between abnormal and normal events to perform the detection and recognition of abnormal events.Thus,the feature representation of the video is extremely important.We use a dynamic image to compress a video clip into a single image while suppressing redundant background,and merging the appearance and motion of the moving object into a single image.Extensive experiments on UCF-crime datasets show outstanding performance of the proposed method in the abnormal behavior detection and recognition.2.We propose deep metric learning for human action recognition with SlowFast networks.We adopt SlowFast Networks to extract slow-changing spatial semantic information of the target entity in the spatial domain,and the fast-changing motion information in the temporal domain.Since deep metric learning is able to learn the class difference between human actions,we adopt deep metric learning to learn a mapping from the original video to the compact features in the embedding space.The proposed network consists of three main parts:1)two branches independently operating at low and high frame rate to extract spatial and temporal features;2)feature fusion of the two branches;3)joint training network of deep metric learning and classification loss.Experimental results on the KTH human action dataset demonstrate that the proposed method has faster runtime and less model size than C3D,R3D,ensuring high guarantees a higher accuracy.