Study On Eliminating Shadow Contamination In High-resolution Urban Land-cover Mapping

In recent years,low-altitude remote sensing platforms(e.g.,UAVs)and their supporting image processing systems are rapidly developing,and thus high-resolution image products(e.g.,orthophotos)become increasing.High-resolution image products have the richer feature information,more accurate spatial relationships,and more flexible collection ways.As primary data sources,these image products have increasingly been applied in diverse fields,such as urban planning and management,resource optimization and allocation,land-cover/land-use information management,etc.However,shadows from tall buildings and tall vegetation are common phenomenon in the spatially heterogeneous landscapes such as high-resolution urban environment.On one hand,shadows can reduce the urban heat island effect providing outdoor thermal comfort,and can serve as a clue for building identification.On the other hand,some shadows that frequently appear in remote sensing images become blind spots for urban sensing due to the relatively fixed position of the sun's altitude and azimuth angle during imaging.They are generally considered a nuisance obscuring details of fine-scale geographic objects and pose a significant challenge for accurate land-cover mapping.Therefore,fundamental differences in the heterogeneity of land-cover types across various urban patterns suggest the need for a generalizable and accurate methods to eliminate shadow contamination in high-resolution urban land-cover mapping.We also note that the most previous studies attempted to recover the ground objects' spectral reflectance to their non-shadow conditions.While recent studies focused extensively on using two sequential steps – shadow detection and shadow removal to eliminate shadow contamination,errors prorogating through steps inevitably raises a new concern of model robustness across high-resolution urban land-cover types.However,shaded landscapes and corresponding homogeneous non-shaded landscapes are continuous.Shadows only reduce the image's signal-to-noise ratio and do not eliminate the original urban scene's semantic association.Based on those considerations,we developed a series of robust models,capitalizing on the state-of-the-art deep learning architectures to eliminate shadow effects in high-resolution urban land-cover mapping.We quantitatively evaluated the model performance in diverse cities and remote sensing datasets across various urban-rural development patterns.In this study,the following summaries can be drawn.Firstly,we created a Shadow Semantic Annotation Database(SSAD)from the collected 1 m resolution NAIP aerial imagery(National Agriculture Imagery Program)for urban shadow understanding in section 3.The SSAD was achieved by transfer learning and Human-computer interaction annotation,which comprises 103 image patches containing various types of shadows along the urban-rural gradient.The size of each image patch is 500 × 500 pixels.Two categories of annotation in the SSAD are(i)shadow annotation(i.e.,shadow and non-shadow)for straightforward shadow detection;and(ii)land-cover annotation,including the six land-cover classes within shadows – building,tree,grass/shrub,road,water,and farmland.Another unique contribution of this study is developing a shadow semantic annotation database – SSAD,which is accurate,diverse,and extendable.Such knowledge can be easily used to help train a deep learning model for high-resolution shadow removal in other urban regions.Secondly,we developed a Recurrent Shadow Attention Model(RSAM)in section 4,capitalizing on state-of-the-art deep learning architectures,to retrieve cast and selfshadows along the urban-rural gradient.The RSAM differs from the other existing shadow removal models by progressively refining the shadow detection result with two attention-based interacting modules – Shadow Detection Module(SDM)and Shadow Classification Module(SCM).The SDM was intended to focus primarily on cast shadows,while the SCM was designed to give more attention to self-shadows.Our results show excellent model performance(F1-scores of 91.1%)for RSAM to extract self-shadows and cast-shadows.Findings suggest that RSAM is a robust solution to eliminate the effects in high-resolution mapping both from cast and self-shadows that have not received equal attention in previous studies.Further results show that RSAM is less affected by the time and perspective of remote sensing images.Thirdly,based on invariable semantic features,we developed a Shadow Semantic Compensation Model(SSCM)in section 5,capitalizing on state-of-the-art semantic segmentation networks,to retrieve fine-scale land-cover classes within cast and selfshadows.SSCM applied the RSAM proposed in section 4 as its shadow attention generator,and directly extracted land-cover information within shadows using SCM.This method provides a feasible and accurate solution for retrieving semantic knowledge in shadows under complex urban scenes,further avoids the dependence on image spectral compensation within shadows.Our results show an overall accuracy of 90.6% and Kappa of 0.82 for SSCM to extract six land-cover categories within shadows.Finally,we applied three competitive experiments to validate the model's robustness and generalization ability in section 6.Findings suggest:(1)both RSAM and SSCM achieved superior performances across 20 urban patterns from the U.S.metropolitan areas,with the high accuracy of 97.06%(F1-scores)to detect shadow,and 90.56%(OA)and 0.82(Kappa)in land-cover mapping within shadows.(2)the results of Vaihingen dataset(9 cm spatial resolution)indicated relatively robust model performances for RSAM and SSCM,revealing 96.33%(F1-scores)to detect shadow,and 89.25%(OA)and 0.80(Kappa)for land-cover mapping within shadows.(3)Considering high-resolution urban shadow contamination,we used deep learning model with SSCM to map the city of Raleigh.If using the proposed SSCM,the deep learning model can achieve higher urban mapping accuracy,such as 90.15%(OA)and 0.87(Kappa).Compared to without SSCM,the model performance can improve by 7% and 10% in OA and Kappa,respectively.The above robust model performances further demonstrated that our proposed models in this study have great application potential in high-resolution urban land-cover mapping and shadow remove.