Visual-based Indoor Scene Recognition And Its Application For Indoor Localization With A Multi-sensor Fusion Approach

With the rapid development of technology and society,people are more inclined to stay in indoor space rather than outdoors.Therefore,understanding indoor scenes around users will facilitate to provide more intelligent and reliable services for them.However,indoor spaces are far more complicated and changeable comparing to outdoor circumstances,which present huge challenges for indoor scene recognition as well as indoor localization and navigation.In recent years,several kinds of convolutional neural networks have been proposed for visual recognition problems such as object recognition,which improved the accuracy to near human level.However,for scene recognition,especially for an indoor scenario,related works are not sufficient.Besides,there are still a lot of problems to be solved in existing works.Firstly,the manual defined global and local feature-based method cannot represent high-level semantic features for indoor scenes,which lead to a poor indoor scene recognition result.Secondly,even convolutional neural networks can be used to learn high-level representations for indoor scenes,they require large-scale well-annotated dataset to train a model from scratch.Thirdly,most of the existing scene recognition methods are based on supervised scheme,methods in an unsupervised manner or semi-supervised manner are lack of investigation.Besides,in the field of indoor localization,although fingerprinting-based methods can achieve fair performance without additional infrastructures in some circumstance.Due to the complexity of indoor environments and the instability of signal propagation,this method cannot obtain stable and accurate localization results in complicated scenarios.Moreover,it is hard to maintain and update the database of fingerprints when changes happened in the localization area.In order to overcome problems in existing methods,this thesis investigated approaches for indoor scene recognition from perspectives both in supervised learning and unsupervised learning.In addition,to further explore applications of indoor scene information for location-based services,scene recognition has been adopted to improve traditional indoor localization system with multi-sensor fusion approach.The main contents and contributions of this thesis can be concluded as follows.(1)Indoor scene recognition methods based on convolutional neural networks have been investigated.From the perspective of supervised learning,different network structures and three training strategies for indoor scene recognition have been analyzed.The first is training an indoor scene recognition model from scratch with a large-scale dataset.The second is fine-tuning a trained model for indoor scene tasks.And the third is taking a trained model as a feature extractor for indoor scene data,and then the indoor scene recognition model can be trained with a simple classifier.In the meantime,a large-scale dataset only for indoor scenes has been extracted from others,which contains millions of annotated images.What's more,methods for further expanding and enriching dataset also have been investigated.On the one hand,this work expands the scope of research for indoor scene recognition problems with large-scale indoor scene dataset.On the other hand,supervised visual representation learning approaches with convolutional neural networks have been investigated and analyzed,which provide a basis for better solving indoor scene recognition problems.(2)An unsupervised visual representation learning method has been proposed for indoor scene recognition.In this method,a k-NN graph has been constructed to mine positive and negative image pairs with constraints in this graph.Then a Siamese ConvNet for binary classification has been adopted to learn visual representations which taken the mined image pairs as inputs.After that,features extracted from the trained Siamese ConvNet can be used to train an indoor scene recognition model with a simple classifier such as linear SVM.The purpose of this work is to learn visual representation for indoor scenes in an unsupervised manner,which can be used to solve problems of insufficient well-annotated training data as well as difficulty in learning high-level semantic features for indoor scenes.(3)A multi-sensor fusion approach for indoor localization based on indoor scene recognition has been proposed.In order to conquer problems in existing multi-sensor fusion approach for indoor localization,this thesis proposed a scheme that adopts scene information as global constraints for indoor localization to decrease the ambiguity of signals.The whole system can be divided into two stages: offline database construction and online localization.In the offline stage,data from built-in sensors of a smartphone would be collected and trajectories of users would be recorded according to different indoor scenes.Therefore,a hierarchical fingerprint database indexed by scene information can be constructed.In the online stage,scene recognition would be conducted first to narrow the area for localization which can decrease mismatching of fingerprints caused by signal instability and ambiguity.The process of fingerprint database construction in the proposed method is more convenient than traditional ways which require a lot of reference points to collect stable fingerprints.It is easy to be updated and extended to a large area.With the proposed method,problems for traditional multi-sensor based indoor localization system can be well solved by scene constraints.And the results also have shown better performance compared to the traditional scheme.In general,this thesis has proposed different methods for indoor scene recognition in both supervised and unsupervised manners,which considering different application circumstances.Moreover,an application of indoor scenes in multi-sensor based localization system also has demonstrated an improvement of localization performance which has shown the potential of indoor scenes for pervasive applications in the future.Therefore,this thesis has great significance both in theory and in practice.