Fusion Of Multi-temporal Multispectral Images And OpenStreetMap Data For The Classification Of Local Climate Zones

Local climate in urban areas has widely caused concerns with the rapid urbanization process.In order to delineate the urban internal structure and further apply it to urban models,the Local Climate Zones(LCZs)concept was proposed.LCZs is a worldwide standardized classification scheme which recognizes urban landscape with respect to local physical structures.Meanwhile,the satellite imaging technique has been rapidly developing in recent years and has become one of the main data acquisition techniques in the remote sensing field.OpenStreetMap(OSM)has become a well-known and free-editable map of the world that provides geographic data from and to citizens.Furthermore,fusing satellite images and OSM data provide much potential for further improving classification performance.Therefore,this thesis fuses multi-temporal Landsat-8 images,single-temporal Sentinel-2 images,buildings layers of OSM data and landuse layers of OSM data for the classification of LCZs.However,LCZs classification is still facing many challenges due to the scheme complexity,unsatisfied knowledge transferability,and features heterogeneity.First,each label of LCZs is defined as a certain combination of some ground types with certain structures arrangement instead of a single object.Second,knowledge trained from some cities may not be transferred well to other cities.Third,satellite images and OSM data are heterogeneous from the aspect of their acquisition types,data meanings,resolutions,and noise sources.Specifically,the aspect of noise sources means that OSM data contains wrong or incomplete recordings,which has become one of the most severe limitations.Because of the features heterogeneity,feature stacking isn't an optimal fusion model even if it is most commonly used in image classification due to its efficiency and fast implementation.This thesis proposes a LCZs classification framework integrating with two novel fusion models,aiming to deal with the heterogeneous feature fusion problem.One model is for fusing satellite images and landuse layers of OSM,and the other model is for fusing satellite images and buildings layers of OSM.Meanwhile,this thesis proposes a simple but effective approach to generating building confidence masks which manage to detect incomplete recordings in buildings layers of OSM.The framework is structured as the ensemble learning approach based on Canonical Correlation Forest(CCF)(Rainforth and Wood,2015)and two fusion models.First,it extracts spectral,spatial,and texture features from satellite images,then it extracts landuse and building features from OSM data.Second,it applies ensemble learning based on CCF and two fusion models during training and testing periods.Third,it post-processes the classification maps and conduct decision fusion.Besides,a baseline framework is defined as a comparison to the proposed framework.It is exactly the same as the proposed framework,except that it applies feature stacking instead of the proposed fusion models when fusing satellite images and OSM data.The dataset was prepared by 2017 IEEE GRSS Data Fusion Contest(Tuia,Moser,Le Saux,et al.,2017).The study cites are distributed worldwide,including five training cities(Berlin,Hong Kong,Paris,Rome and Sao Paulo)and four testing cities(Amsterdam,Chicago,Madrid and Xian).Experimental results indicate that the proposed framework is effective in dealing with multi-source data fusion for LCZs classification with high generalization capability on a worldwide scale.First,the highest accuracies(OA is 76.15 % and Kappa is 0.72)of the proposed framework outperform the accuracies of the baseline framework(OA is 70.14 % and Kappa is 0.65)by 6.01 % and 7% respectively,and also outperform the first place of 2017 IEEE GRSS Data Fusion Contest(Tuia,Moser,Le Saux,et al.,2017)by 1.21 % and 1 % respectively.Second,the proposed framework has better temporal robustness.Temporal robustness means the classification performance should be comparable when using various single or multi-temporal satellite images from one or more acquisition times.For example,after applying classification models to single temporal Landsat-8 images at each acquisition time in Amsterdam,the accuracies of the proposed framework(OA ranges from 72.59% to 81.88 % and Kappa ranges from 0.68 to 0.78)are more stable than those of the baseline framework(OA ranges from 35.6 % to 64.57 % and Kappa ranges from 0.3to 0.6).In addition,the proposed framework requires less temporal information to achieve the comparable classification performance due to its temporal robustness.This thesis leaves many open questions for future studies.First,knowledge transferability between cities is still not optimal after applying CCF and the proposed fusion models.It is worth to explore more advanced approaches such as domain adaptation,semi-supervised classification,active learning,etc.Besides,it is still not clear that which kinds of knowledge is worldwide transferable considering diverse city constructions.Second,multi-source data brings up more information but meanwhile causes data compliance problem from the aspects of features or acquisition times.It needs efforts to derive a more advanced model which could optimally collaborate all information.