| Point cloud and image segmentation has always been a hot area of computer vision,and deep learning has also promoted the research and progress of point cloud and image semantic segmentation.However,the current segmentation model based on deep learning cannot guarantee the accuracy in a new domain with inconsistent data distribution,which needs the label data to be retrained to adapt to the new domain.This method requires a lot of manpower and material resources for sub-level labeling.Therefore,it is of great practical significance to study the Unsupervised Domain Adaptation(UDA)method that transfers the labeled source domain knowledge to the unlabeled target domain.But most UDA methods act on image modalities,and very few directly act on point clouds.Therefore,this article has done the following work on the above issues:1.This paper first uses the cross-modal UDA(xMUDA)method,through the image point cloud semantic segmentation network imitating each other’s way of learning,and enhancing the semantic segmentation performance of each modal in the target domain.The attention mechanism based on gate control is introduced into the image feature extraction network,which enhances the feature extraction of some stable information in the autonomous driving scene.On the condition of improving the semantic segmentation accuracy of the image modal in the target domain data,the point cloud semantic segmentation accuracy is synchronously improved to achieve a better domain migration effect.2.After analyzing the shortcomings of the existing xMUDA method,this article introduces a adversarial-based method to the original algorithm.On the basis of the xMUDA method,consider aligning the feature difference between the source domain and the target domain in the feature space,and through the adversarial training the difference between the two point cloud semantic segmentation heads,a better point cloud semantic segmentation task decision boundary is learned.This paper conducts domain adaptation experiments on commonly used autonomous driving datasets to verify the algorithm of this paper.The cross-domain dataset includes three types: cross-city domain dataset,cross-day and night domain dataset,and cross-sensor domain dataset.The results show that the algorithm in this paper has achieved the best semantic segmentation results in the three sets of cross-data set point cloud and image semantic segmentation experiments.Compared with xMUDA,the overall improvement is up to 0.3,2.8 and 3.3 m IOU,and the improvement ratio is 0.4%,5.6% and 7.7%.Among them,the main categories,such as vehicles,increase by up to 2.2,1.9 and 4.8 m IOU,which realizes the Lidar point cloud semantic segmentation domain adaptation fusing image for urban scene. |
