Research On Human Action Recognition In Infrared Videos

Action recognition is an important task in computer vision with a wide application prospect and considerable economic benefits.Current research efforts mostly address action recognition in visible imaging videos,and new technologies and methods have been put forward,making breakthroughs in recent years.Nevertheless,few works address the action recognition task in infrared spectrum.Infrared videos not only suit for situations which require monitoring around the clock,but also protect privacy.Therefore,recognizing action in infrared videos is of important research value.To this end,this thesis focusses on the task of action recognition in infrared spectrum.The specific studies are as follows:This thesis makes a full investigation of the visible action datasets and an infrared action dataset,namely,the Infrared Action Recognition Dataset(InfAR).Inspired by the construction approach of existing visible datasets,this thesis expands greatly the InfAR dataset and evaluate the new dataset.Both of the new and original infrared video clips are taken from varying real-world scenes.Imaging factors like background,occlusion,viewpoint and season are taken into account.Moreover,conventional methods in action recognition are selected and compared to make a comprehensive evaluation on the new dataset.Currently,most of the action recognition methods extract features on the whole image,which are likely to be mixed with additional interference information,leading to a low recognition accuracy.To solve this problem,this thesis proposes a novel infrared action recognition method based on saliency regions.Firstly,this method extracts optical flow-motion history image(OF-MHI)features from the image sequences,obtaining the motion information of videos.Then,the class activation map(CAM)method is applied to eliminate the interference from moving targets to get saliency regions.Thus,the significant regional features can be obtained.Finally,the convolutional neural network(CNN)is utilized to extract the final features from the significant regional ones,and then the CNN features are fed into the support vector machine(SVM)to obtain the recognition results.Experimental results show that the proposed method can effectively improve the recognition accuracy compared with the traditional methods.Furthermore,conventional two stream CNN methods face the problem of unreasonable appearance and movement information usage.Therefore,this thesis proposed an improved two-stream action recognition method which contains a pose channel and a motion channel.For pose channel,the skeleton features are extracted as appearance information.For motion channel,the original optical sequences are computed as motion information.Then,these two features are fed to C3 D model respectively to obtain appearance and motion CNN representations.Then the late fusion model is applied to fuse dual CNN representations.Finally,fused representations are utilized for action recognition though a SVM classifier.Experimental results show that this method achieves a higher recognition accuracy than the existing two stream method.