Research On Panoptic Segmentation Method Of Remote Sensing Image Based On Combined Processing And Instance Augmentation

Panoptic segmentation of remote sensing image aims to assign each pixel in the image to different ground object categories while assigning identity to each object within each category.This technique holds great potential for various applications in many fields,such as land resource exploration,marine,agriculture and forestry monitoring.However,compared to general image panoptic segmentation,panoptic segmentation of remote sensing image presents greater challenges due to the more complex ground object scenes and the smaller distinctiveness among different ground object categories.Traditional panoptic segmentation methods of remote sensing image often divide the task into instance segmentation and semantic segmentation tasks for separate processing,leading to increased model complexity and additional computational overhead.Furthermore,when applying the panoptic segmentation method of universal image based on masked-attention mask Transformer to remote sensing image,there are notable challenges related to negative samples affecting segmentation accuracy and repeated instance predictions.To address these problems,this thesis introduces the combined processing methods of fully convolutional network and mask Transformer respectively to jointly handle instance segmentation and semantic segmentation tasks;introduces the data augmentation method of cropping with instance to solve the negative sample affecting problem;introduces the Matrix NonMaximum Suppression(Matrix NMS)to optimize the segmentation performance.The main work and contributions are as follows:(1)A panoptic segmentation method of remote sensing image based on fully convolutional network is presented.Aiming at the separate processing problem of semantic and instance segmentation,a unified fully convolutional workflow is employed to represent and predict object instances and semantic regions.This approach eliminates the need for positioning of bounding box or instance separation,making it simpler and more efficient compared to traditional methods.Experimental results on the publicly available BSB Aerial Dataset demonstrate that this method achieves a 0.957% improvement in panoptic segmentation performance,specifically measured by the Panoptic Quality(PQ)metric,compared with the baseline model.(2)A panoptic segmentation method of remote sensing image based on mask Transformer and instance augmentaion is presented.Aiming at the negative sample affecting problem in panoptic segmentation method of universal image based on masked-attention mask Transformer,a data augmentation method is introduced.This method involves obtaining the center coordinates of the bounding box of the object instance and randomly selecting coordinates of the cropped points and the size of the cropped area,ensuring that the cropped area contains at least one object instance.Experimental results on the publicly available BSB Aerial Dataset demonstrate that this method achieves a 0.502% improvement in panoptic segmentation performance,specifically measured by the PQ metric.(3)A segmentation performance optimization method based on postprocessing with Matrix Non-Maximum Suppression(NMS)is studied.Aiming at the repeated instances problem in the current method when generating panoptic segmentation predictions,a post-processing module of Matrix NMS is introduced.This module recursively reduces the confidence scores of other masks based on the intersection over union ratio,filtering them out with a smaller threshold,thus retaining the highest-scoring predicted instance.Experimental results based on the publicly available BSB Aerial Dataset demonstrate that by introducing the post-processing module of Matrix NMS,the two methods achieve improvement of 0.714% and 0.886% respectively in the instance segmentation metric,specifically measured by the Average Precision(AP).The thesis also presents the design of a panoptic segmentation software for remote sensing image.This software incorporates the methods presented in the thesis,and the application results demonstrate a certain level of generalization ability.