Thermal Image Semantic Segmentation Based On Convolutional Neural Networks

The visible image is obtained by external light which shoots at the surface of the object first and then reflects,the thermal infrared image is formed by thermal infrared light emitted by the target itself,so the thermal infrared camera can capture images effectively in the dark.Therefore,thermal semantic segmentation can achieve image analysis and understanding at any time.The thermal infrared image is essentially a representation of the temperature field of the observed object and its surrounding environment,and has the characteristics of low resolution,weak contrast,weak texture,and inconspicuous contour.Using visible semantic segmentation algorithms directly cannot segment thermal infrared images well.This thesis conducts related research on how to solve the problem of thermal infrared semantic segmentation,which not only achieves good segmentation accuracy but also meets the requirements of real-time.The main research results of this thesis include the following three aspects:First,an edge-guided network based on gate mechanism is proposed to achieve precise thermal semantic segmentation.Aiming at the problems of weak contour and weak contrast in thermal infrared image,the edge information is used as a priori knowledge and edge guided module is proposed to make the network better segment the target.Since the additional edge information introduced may contain noise and interfere with the segmentation,a gate mechanism is added to suppress the noise.This method achieves good results on thermal semantic segmentation dataset SODA,proving its effectiveness and superiority.In addition,it has also been verified and obtained good accuracy in the visible-based dataset,proving that the method is also applicable to visible-based dataset.Second,we established a thermal infrared image semantic segmentation dataset.Due to the lack of effective public thermal infrared image semantic segmentation dataset,we made a thermal infrared image semantic segmentation dataset "Segmenting Object in Day And night" referred to as "SODA",and provide publicly available for academic research.The dataset contains dense pixel labeled images and virtually generated images.The dense pixel labeled dataset has 20 common categories and the collected images are mainly from multi-angle in campus scenes.Due to the large and heavy workload of manual data collection and images labeling,we also use image translation method to convert the existing publicly available visible image semantic segmentation dataset into simulated thermal infrared modal dataset.The final SODA contains two parts,one is manually labeled,and the other is generated by the algorithm.Third,a real-time thermal infrared image semantic segmentation network based on MobileNetv2 is proposed.Although the deep convolutional neural network can obtain good segmentation results,its parameters are large and the test stage is time-consuming.If it is deployed on a mobile terminal,it cannot meet the requirements of low storage and real-time.Therefore,MobileNetv2 is selected as a lightweight backbone network,and the features generated by MobileNetv2 not only need little calculation parameters but also are discriminable.In order to further reduce the number of parameters,in the final stage of thermal infrared image semantic segmentation,the segmentation head structure is improved,and a lightweight multireceptive field head structure is proposed.Finally,experiments on the thermal infrared image semantic segmentation dataset SODA show that although the accuracy of proposed method is slightly reduced,it achieves real-time running speed and low parameters.