Visual Inertial SLAM System Based On Long-Wave Infrared Cameras

Visual simultaneous localization and mapping(SLAM)has developed rapidly in recent years,applications and research results have been widely used in the fields of mobile robots,autonomous driving,and virtual reality.In an ideal environment,visual SLAM system perceives the texture information in the scene with visible light camera for pose estimation,establishes a map of surrounding to complete localization request in an unknown environment.However,the existing visual SLAM algorithms are prone to environments.Visible light camera cannot perceive effective texture information in deteriorated illumination or smoke environments,resulting in the failure of localization and mapping.Long-wave infrared camera,also named as thermal camera,perceives long-wave infrared information,which the imaging mechanism is related to the radiation of the scene.The thermal camera is robust to illumination changes in visible light and is an ideal visual sensor for day-and-night positioning and mapping.The thermal camera has a similar projection model to the visible light camera.However,existing visual SLAM algorithms have poor localization performance in the infrared domain.Thermal images have defects such as fixed pattern noise,low contrast,continuous frame photometric changes,periodic data loss,et al.Existing visual SLAM algorithms are challenging to utilize thermal information directly.The excellent positioning and mapping performance of visual SLAM is based on high-quality input data.Converting defective thermal data into stable and high-quality image data is a breakthrough to achieve thermal visual SLAM.Thanks to the rapid development of computer performance,deep learning has performed well in various fields in recent years.Deep learning shows powerful data association capabilities.Combining traditional SLAM technology with deep learning is a feasible way to achieve robust thermal visual localization and mapping.This dissertation proposes a state-of-the-art visual-inertial SLAM system based on thermal cameras.The system solves feature extraction and association problem on thermal images,and has advanced and robust localization accuracy in both indoor and outdoor environments with different scales and lighting conditions.The main research contents and innovations of this dissertation are as follows:Firstly,aiming at the quality defects of thermal images,a thermal image processing method based on singular value decomposition(SVD)is proposed.The proposed method starts from the imaging mechanism of the long-wave infrared camera,removes the largest singular value of the infrared image,and reconstructs the singular value of the image.The method effectively solves the problems of high noise,low contrast and photometric change,the existing feature extraction algorithms have excellent feature extraction performance in the processed thermal images.Secondly,a real-time lightweight thermal optical flow network is proposed for thermal data association.The training stage uses a self-generated thermal optical flow dataset,and the trained network can achieve accurate optical flow estimation of thermal images.A thermal visual inertial odometry system is designed and implemented based on the thermal image processing method and proposed optical flow network,and the camera poses and map are optimized by sliding window.Experiments illustrate that the odometry system can achieve accurate and robust pose estimation in indoor multi-scale and variable illumination environments.Thirdly,Aiming at the cumulative error caused by the long-term operation of the odometry,a loop closure module is added and a complete thermal visual inertial SLAM system is proposed.The system optimizes the keyframe selection and management mechanism,and utilizes a strict strategy for selecting feature tracking results to reduce the error of the SLAM front end.The loop closure is designed for thermal images and has excellent loop detection and correction performance,ensuring the system’s globally consistent localization and mapping.Evaluation results show that the proposed system has state-of-the-art localization performance in indoor and outdoor scenarios.