Research On Pedestrian Detection And Semantic Segmentation Algorithm Based On RGB And Infrared Fusion

Using advanced visual algorithms to improve the environmental perception ability of robots is a hotpot aspect in the field of computer vision.However,there are many limitations in visual perception algorithms based on visible information.Thermal infrared cameras can obtain clear environmental information in the dark,which is of great help to improve the perception ability of robots.Therefore,this paper studies the perception algorithm based on the fusion of visible and thermal images.A multispectral pedestrian detection model and semantic segmentation model are designed by using the deep learning technology.Besides,the designed algorithms are deployed on the six-wheel unmanned platform to improve its environmental awareness.The detailed contents are as follows:1.Aiming at the high miss rate and slow speed of pedestrian detection based on visible images in poorly illuminated environments,an end-to-end multispectral pedestrian detection model is constructed based on multispectral fusion technology.Firstly,the multispectral pedestrian detection model is constructed,and the spectral feature fusion module and the high-dimensional feature fusion module are designed to fuse different spectral information.Secondly,the performance of the designed model is evaluated on the KAIST dataset.Finally,ablation experiments are designed for the fusion modules to prove the effectiveness of our method.2.Aiming at the problems of low segmentation accuracy and slow speed of RGB-based semantic segmentation models in insufficient illumination,a lightweight multispectral semantic segmentation network is constructed by using multispectral feature fusion technology.Firstly,a multispectral semantic segmentation model is designed based on the encoding-decoding structure,and the Skip Inception module is constructed for high-dimensional feature extraction.Secondly,the performance of the designed model is evaluated on the multispectral semantic segmentation dataset.Thirdly,the semantic segmentation model is applied to solve the problem of multispectral flame segmentation.Finally,ablation experiments are designed for the Skip Inception module to prove its effectiveness.3.Aiming at the problem that the six-wheel unmanned platform has poor environmental perception ability in the dark environment,the multispectral pedestrian detection model and semantic segmentation model proposed in this paper are deployed on this platform to evaluate its performance in the actual campus environment.Besides,the models are adjust based on the campus environment to improve the environmental perception ability of the platform in the dark.