Panoramic Image Segmentation And Visual Enhancement Based On Semantic Segmentation

In the parking process,the driver needs to observe the parking space and surrounding vehicles and other obstacle information through the rearview mirror.The limited observation perspective is not conducive to the driver to fully obtain the surrounding environment information,which often leads to the failure to reverse into the garage or the scraping with other vehicles in the parking process.In the automatic parking system,it is also necessary to collect the lane line and parking line information around the car body,so as to safely and smoothly control the vehicle to park into the parking space according to the predetermined route.Panoramic image can collect the environmental information around the vehicle,including the driving line,parking line,other vehicles,pedestrians and other obstacles,which can directly help the driver avoid danger in the process of parking and park smoothly.These information can also be used as the input of the automatic parking system to obtain the best parking path through the comprehensive judgment of the system.However,the bird ’s-eye image of the vehicle is severely distorted and incomplete,and the visual information becomes very blurred in some poorly lit environments.If the driver cannot see the surrounding environmental information,the risk of collision will increase,and in the automatic parking system,it is more dangerous to fail to obtain the comprehensive and true environmental information.In order to make better use of the panoramic image to perceive the local environment,we use semantic segmentation algorithm to process the vehicular panoramic image to obtain key information such as parking space line and driving line.Meanwhile,based on the results of panoramic image segmentation,a virtual panoramic monitoring system is built to provide clearer perceptual information for drivers.The main research contents include:(1)A lightweight semantic segmentation network,Segnet＿DS,is redesigned based on Segnet.The new network model uses the idea of deep supervised learning,adding extra output branches to the hidden layer of the codec,and using all branches to calculate the training loss function,so as to fully supervise the learning of the whole network layer.All supervisory branches remain independent from the trunk network and participate only in the training process,not in the reasoning process.This model improves the segmentation accuracy without increasing the inference time.(2)More panoramic images are collected and annotated in this article.In order to better train the model proposed in this paper,the panoramic image data sets are collected and annotated by ourselves.According to the image transformation environment existing in the panoramic image,the corresponding image enhancement method is designed.The new data set and the targeted image enhancement method further improve the generalization performance of the proposed model.(3)Visual enhancement is performed for panoramic segmentation results.The experimental model proposed in this paper is compared with the latest panoramic image segmentation method and the latest lightweight semantic segmentation model on the public data set,and the results show that the proposed model achieves a relatively ideal result in the reasoning speed and segmentation accuracy.Finally,in order to make it more convenient for the driver to observe the key road surface information around the vehicle,a panoramic enhanced visualization result based on segmentation results was established in this paper.Experiments have proved that compared with the current state-of-the-art methods and lightweight semantic segmentation models,the proposed model has high segmentation efficiency and can effectively overcome the occlusion,damage,and blur problems of road signs in the around view monitor system.Besides,it also meet the real-time requirement.