The Study On Theme Classification Of Web Map Service By Fusing Textual And Visual Descriptions

Maps have been widely used in various applications,such as traveling,public health,urban planning and management.Web Map Service(WMS)works as a popular specification in promoting maps sharing and interactions.There have been massive WMS maps shared on the web.However,most maps and their rich inner information are left alone except that less of them are used as basic maps.Therefore,we try to mine their values by classifying their themes,which could provide clear and unified themes for maps retrieval and other theme-based applications.In addition,we hope that our work could inspire others in mining more valuables information from existing massive WMS maps.There have been many papers published on recognizing or classifying map themes,and they have proposed various effective features and methods.However,most of them only use the title,keywords and abstract of the target map,which ignore other potential textual description related to the map theme.Besides,maps are also used to classify maps,but existing research only use them to classify those themes with significant visual features.To the best of our knowledge,no people pay attention to model the complex relationship among general map themes and visual features.At the same time,few papers explore in how to combine the textual and visual descriptions to classify map themes.To handle above problems,we try to make a full use of the theme-related description within WMS.Some strategies are proposed to combine the potential theme-related descriptions,and some new features are also designed to capture and represent them.Besides,a latent feature based model is proposed to fuse aforementioned multimodal features to classify map themes.Finally,the experiment results answer how multimodal features work together to improve the classification performance over unimodal features.Specially,it consist of four detailed parts:(1)Supervised Feature Extraction on Enriched Textual Description using Inner Sources. To handle challenges caused by confusing or missing description from maps' metadata fields(title,keywords,and abstract),an enrichment and analysis framework on WMS textual description is proposed,which could embrace the outer and inner sources.Here,we focus on the inner sources,where three types of descriptions are selected to enrich maps'metadata fields.It provides an effective way to fill in the missing content.In addition,we import the supervised term weighting schema into text analysis on maps and modify it as RTF-IC(9, which could capture the label information from training data.The experiment results verified the effectiveness of the enhancement strategy and the modified feature.(2)A Multi-level Framework for Visual Features of Maps and Legends.To model the complex relationship among general map themes and visual features,we propose a multi-level framework to present the visual features from maps and legends.In the framework,some new visual features are proposed,including map symbol types, legends'visual features.Besides,we optimize the popular pre-trained Inception- Resnet-v2 features for maps and legends using PCA.Comparing to the traditional single-level features on maps,the framework could significantly improve the classification accuracy on general themes.(3)Latent Features based Multimodal Features Fusion for Theme Classification.To fuse the heterogeneous multimodal features,a latent features based fusion strategy is proposed to make full use of multimodal features instead of the decision-level fusion.The complex computation of latent features and the classification model is solved by a deep neural network model with end-to-end training.The multimodal classification model outperformed unimodal models on the them classification accuracy,and it could also keep high accuracy with missing metadata fields.(4)A Multimodal Dataset for Map Themes.To explore testify the effectiveness of multimodality fusion on map themes,a collaborative online graphical user interface is developed to facilitate the multimodal dataset labeling.As a result,a multimodal dataset for the INSPIRE themes on WMS maps is created,which consists of metadata text,maps,legends,and themes.