Application And Real-time Image Semantic Segmentation Based On Deep Learning In 3D Reconstruction System

In recent years,the rapid development of artificial intelligence technology has provided new ideas and directions of development for the autonomous movement of UAV and unmanned intelligent driving.Most UAV and unmanned achieve obstacle avoidance relying on laser radar which's accuracy is high,but it has the disadvantages of large size,high power consumption,high price and the inability to work in extreme weather such as rainy days.UAV and unmanned rely on GPS which can provide the two-dimensional latitude and longitude coordinates all day long,but it couldn't give specific 3D spatial information of the target.At the same time,it has the disadvantages of low civilian positioning accuracy and inaccurate positioning under the overpass.In order to solve these problems,people try to use the deep learning and visual techniques such as semantic segmentation and 3D reconstruction to build a low-cost,high-precision and obstacle-avoiding navigation system,which promotes the development of UAV autonomous movement and unmanned intelligent driving technology.The paper focuses on the research and application of image semantic segmentation algorithms for terminals and backends.Firstly,the basic principles of mainstream image segmentation algorithms which include MNC,Mask-RCNN,FRRN and DeepLabv3 are studied.The in-depth analysis and comparison are carried out,and the segmentation accuracy and segmentation speed are comprehensively evaluated on high-performance computing devices.Based on the embedded heterogeneous computer system structure and on the basis of evaluation,the real-time image semantic segmentation algorithm ENet that satisfies the mobile embedded end is studied.The idea of the current performance-leading image semantic segmentation algorithm Mask-RCNN which is based on the Faster R-CNN target detection,and add the mask branch to each ROI for pixel-level segmentation prediction is absorbed.The real-time embedded terminal image semantic segmentation algorithm based on Yolov3 + Enet is proposed.And the algorithm is performed and optimized on the embedded heterogeneous platform NVIDIA Jetson TX2.The neural network model performs model's compression,pruning and quantization operations to achieve the goal of greatly improving the speed by sacrificing a small amount of precision.In addition,based on the kitti city street view dataset the algorithm is trained to gain the semantic segmentation model,and which's speed and accuracy performance are analyzed and evaluated.The high-precision semantic segmentation at the back end was focused to complete binocular calibration,binocular stereo matching and parallel optimization of the algorithm on TX2 mobile platform.Finally,building a semantic 3d SLAM Map based on ORB-SLAM+DeepLabv3.