Research On Mars Terrain Segmentation Based On Deep Learning

With the development of Mars exploration missions in many countries,intelligent sensing technology for Mars environment have received extensive attention in recent years.Mars terrain segmentation aims to assign all pixels of an input image with various terrain labels,which provides a firm support for the downstream research on rover traversing and geologic analysis tasks.Compared with traditional methods that rely on manual features,deep learning-based Mars terrain segmentation research has achieved more outstanding performance in the past two years.However,the current research in this field is easily limited by datasets,and there are still some shortcomings in method design.On the one hand,the existing Mars terrain segmentation datasets suffer from insufficient sample size,imprecise classification,rough labeling and single purpose,which are difficult to train deep networks in various Mars terrain segmentation missions.On the other hand,many issues in Martian terrain segmentation methods have not been effectively solved.First,it is difficult for existing deep learning methods to take into account the extraction of local features and global context of Mars rocks,resulting in poor robustness of the trained network for Mars rocks with different appearances.Second,existing supervised learning methods that excel in structured Earth scenes overlook the exploration of the relative relationship between adjacent heterogeneous terrains on Mars,which need to be improved to be more suitable for multi-terrain segmentation tasks in unstructured environments such as the Martian surface.Third,existing multi-terrain segmentation methods rely too much on data annotation and have poor generalization ability in the face of multiple Mars datasets with great cross-domain differences.Transfer learning technology is needed to learn robust cross-domain knowledge among different Martian datasets.Therefore,the paper conducts research on Mars terrain segmentation based on deep learning,providing reference intelligent solutions for existing and future Mars exploration missions.The main research content and contributions of the paper are as follows:(1)Aiming at the issue that the existing Mars terrain datasets are difficult to support the training of deep networks,a Mars panorama segmentation dataset named MarsScapes and two Martian rock segmentation datasets called MarsData-V2 and SynMars are constructed to provide a reliable standard for evaluating Mars terrain segmentation methods.MarsScapes contains 195 panoramic maps of the Martian surface,and provides semantic and instance segmentation labels,which can be used to evaluate both supervised network and unsupervised domain adaptive method.MarsData-V2 annotates the rocks in the Mars close-up image,which meets the data requirements of the rock segmentation network after data augmentation.SynMars used Blender to simulate the environment of Zhurong and took 30,000 samples.(2)Aiming at the issue that the existing U-Net networks are difficult to capture the global context of Mars rocks and Transformers tend to flooding local information,a multi-layer coding-decoding network based on U-Net and Transformer is proposed,called Rock Former.Rock Former extracts multi-scale feature maps with an improved vision Transformer model,which extracts rich local information and global context.In addition,a feature refining module(FRM)is designed to enhance the representation ability of multi-scale feature maps.Experiments were carried out on MarsData-V2 and SynMars datasets.The results show that the method in this chapter has significant improvement on the evaluation indexes of Mars rock segmentation.(3)Aiming at the unstructured characteristics of Martian terrains,an improved supervised learning method based on hybrid attention mechanism(HASS)is proposed for Martian multi-terrain semantic segmentation.The HASS introduces a parallel local inter-class attention branch(LICAB)based on the existing global attention network,which explores the relative relationships between adjacent heterogeneous terrains as an important supplement to the features extracted by the global attention network.In addition,the paper designs an attention merging module to summarize the features extracted by the two branches.The experiments conducted on MarsScapes and AI4 Mars datasets have demonstrated that HASS has better performance in Martian multi-terrain segmentation.(4)Aiming at the issue that existing Mars multi-terrain segmentation methods rely heavily on the amount of data annotation and are difficult to generalize,an unsupervised domain adaptive(UDA)framework based on knowledge distillation and Transformer is proposed,called UDAFormer,which realizes transfer learning between different Mars datasets.The framework transfers the knowledge learned from the source domain to the target domain through a teacher-student model,and alleviates the interference of unsatisfactory data augmentation operations through a regularization layer.In addition,an output-guided biased sampling(OGBS)strategy is proposed,which guides the teacher-student model to pay more attention to similar categories that are easily confused and rare categories that are easily submerged.The experiments were carried out on two tasks,MarsScapes-A→MarsScapes-B and MER-Seg→MSL-Seg.The results show that UDAFormer has significant advantages in the domain adaptive segmentation of Martian terrain.