Object Detection And Localization By Stereoscopic Vision System Based On Thermal-visual Cameras

Fire-reconnaissance robot can replace fireman to enter dangerous scene such as flammable,explosive,toxic,smoke and anoxic,it is an effective way to reduce casualties of firemen.The camera is one of the main sources to get the external information on the robot,it can speed up the rescue work on the scene by analyzing the captured images.Most of the existing methods of analysis are done manually,which not only wastes the valuable manpower,but also can not quickly extract the key information provided in the image.The stereoscopic vision system based on thermal-visual can full use of the gray information provided by the visual camera and the thermal radiation information provided by the infrared thermal camera.In this paper,a stereoscopic vision system based on thermal-visual cameras is designed to realize image segmentation,Parallax matching,stereo vision.The main works are as follow:The shortcomings of traditional Otsu for thermal radiation image segmentation are analyzed.Based on the probability distribution of thermal radiation image histogram,a Otsu threshold segmentation method based on Gauss distribution is proposed.First,one dimension Gauss curve is used to fit the thermal radiation histogram and the segmentation behavior is determined by judging the good or bad of the fitting curve.This method not only solves the error segmentation of the traditional Otsu to the single peak histogram,but also can effectively segment the multi target occlusion.This paper analyzes the applicability of the parallax matching algorithm to the vision system in this paper.Based on the contour characteristics of objects in thermal radiation and visible images,a parallax matching method based on Snake model is proposed.In this method,the result of thermal radiation image segmentation is used to extraction of target contour,which is used as a template to facilitate the gradient image in a certain range of parallax,and the target coordinates in the visible image is calculated by the image force energy function of the snake model.Experiments show that this method can match the same target in the visible light image in a complex background.According to the demand of the system,the existing camera calibration methods are analyzed and selected for the Zhang Zhengyou calibration method for stereo calibration,and the chessboard is made to calibration thermal-visual cameras.The experiment shows that the intrinsic matrix,rotation and translation vectors calibrated by the chessboard can be used to locate the target.In order to simplify the design of parallax matching algorithm,the output image of row alignment is calculated by Bouguet algorithm.