Research On Abnormal Action Recognition Method Based On Infrared And Visible Image Fusion

With the development of computer vision and artificial intelligence technology,intelligent monitoring system is widely used in security neighborhood.It can automatically identify abnormal actions and improve the monitoring efficiency and recognition accuracy.Intrusion action and climbing action are abnormal action that interacts with a scene.The detection of the two actions can detect abnormal conditions in time and reduce the occurrence of dangerous and malignant events.There are limitations to using a single type of sensor for monitoring systems.Under the conditions of low illumination and low visibility,the complementary characteristics of infrared and visible images should be utilized to identify the action associated with the scene by using the information in fused images.When the position of the monitoring camera is fixed,there are differences in the scale,visual angle and target characteristics of visible and infrared imaging.As a result,the efficiency and accuracy of the existing registration and fusion methods based on feature matching are limited.In this paper,a fusion method of infrared and visible images from different views based on saliency detection is proposed.The target of infrared image and the scene details of visible image are fused into one image to realize accurate target positioning.Firstly,by presetting heat sensitive target,the transform model of visible and infrared views is calculated,and the infrared and visible views are registered in advance.Then,the salience region of the pedestrian target in the infrared image was extracted using the Mask R-CNN network,and each infrared target was locally fused with the visible image according to the views transform model points.Finally,the method is validated by the identification of illegal intrusion action.Aiming at the problem of target pedestrian's climbing action recognition under the condition of low illumination,the method proposed in this paper combines infrared and visible image fusion with behavior recognition,and detects abnormal climbing actionrs according to the pose information obtained from human skeleton extraction and the target position information obtained from view transformation.Firstly,the position information of people in the scene is obtained preliminarily according to the view transformation model of infrared and visible images.Then,the single person skeleton information was extracted from the targets detected by Mask R-CNN,and the deep neural network was used to extract features and classify them.Finally,the climbing actions are identified and fused according to the target position and attitude information.Experiments show that the method proposed in this paper can effectively fuse the thermal target information of infrared image with the visible light scene,and can accurately and timely determine whether abnormal climbing action occurs.