Study On Scene Identification And Registration Technology For Mobile Augmented Reality

Augmented Reality (AR) technology is one of the hot topics in the area ofcomputer applications. AR technology can merge virtual objects with the real scenesto enhance the user’s understanding and experience of the real world. AR combinesmany subjects including virtual reality, computer vision, computer graphics, imageprocessing, pattern recognition and opto-electronic display, and has been successfullyapplied in various domains such as industry, medicine, media, defense, entertainment,education, etc.Mobile Augmented Reality (MAR) refers to the augmented reality systemsrunning on mobile devices such as IPADs, smart phones, portable computers, etc.Most conventional augmented reality systems use desktops and workstations as theoperating platform, and such systems limit the scope of activities of the user andcannot be applied to the outdoor environment. The rapid development of mobiledevices and network technology releases augmented reality technology from therestrictions of PC, workstation and other heavy equipment, promoting the emergenceand development of MAR.An augmented reality system is composed of modules such as video dataacquisition, image recognition,3D registration, and fusion display, where imagerecognition and3D registration are some of the core issues to be addressed. Imagerecognition and3D registration based on artificial markers have already come to itsmaturity, but the technology is somewhat limited in its application due to the fact thatartificial markers with special geometries must always appear in the vision. Trackingand registration technology based on natural features is attracting much attentionbecause of its flexibility, however, there are usually no known or predictablegeometries in natural features. This greatly increases the difficulty of theirextraction and matching, affecting the accuracy and speed of the tracking andregistration.Over the past decade, outstanding algorithms for feature extraction and matchingcontinue to emerge, including SIFT (Scale-Invariant Feature Transform),SURF(Speed Up Robust Feature), RT(Randomized Trees), etc., which provide strongsupport for solving the problem of tracking and registration. However, there are stilldrawbacks in the existing algorithms. Most feature extraction and descriptionalgorithms require expensive computation, large memory, and long matching time,making it difficult to achieve satisfactory tracking and registration results. Currently there still exists a gap between the performace of the mobile terminal and that of thegeneral personal computer, mobile terminal has such a special hardware architecturethat the feature matching and registration algorithms designed for the personalcomputer platform is not applicable. This thesis focuses on the research of sceneidentification and registration techniqures that have to be solved when developingMAR system in natural complex environments according to the characteristics ofmobile terminal hardware. The main research contents are listed as the followings:(1) A new approach is proposed to achieve a more effective way to extractfeatures and describe keypoints than the traditional algorithms such as SIFT、SURFfor smartphones with limited hardware resources and weak computing capacity. Afteranalyzing the local feature extraction algorithms proposed in recent years, a newmethod is proposed to detect feature and locate the position of keypoints forsmartphones. At the same time, the distinguish ability of the local features isimproved by fusing local descriptors simultaneously with gravity sensor information,which solves the problem of false matches under the scenes of similar textures.(2) A fast matching algorithm based on binary feature descriptor is proposed torealize scene recognition. Firstly, binary descriptors are segmented to establish indextable of segment descriptors to achieve the quick search of similar descriptors;Secondly, based on the binary feature vectors using Hamming distance to measure thesimilarity, the parallel computing is used to improve the matching speed of segmentdescriptor, then the space information of binary feature vector is used to rearrange thematching descriptor and the sensor information of the binary feature is used to filtermismatch keypoints, which improves the matching accuracy.(3) By doing some research on the problems of large-scale image recognitionappeared in the MAR, A novel scene identification method which combines therandom clustering forests and machine learning algorithm is proposed by studying theproblems of large-scale image recognition in the MAR,. The proposed methodreduces the training time for large scale images with the improved random clusteringalgorithm and increase the accuracy of the same descriptors assigned to the samecluster node by adding the sensor information to the cluster nodes. The SVMclassifiers are trained by collecting images of the scenes with different perspectivesand different scales to achieve highly accurate scene recognition. Meanwhile, in orderto narrow the search range of scene recognition, the geographic information is used todivide the large-scale complex environment into several regions, which further improve the scene recognition rate of the algorithm.(4) Marker based tracking technology is currently the most sophisticated andwidely used techniques to register for augmented reality system. After analyzing theexisting tracking and registration methods based on markers, a tracking andregistration method based on QRcode is proposed to achieve fast registration of thescene by tracking the black box of QRcode and using the global homography relativeto the initial image to calculate the actual pose of the camera. The proposed methodcan effectively improve the stability and accuracy of the registration of the camera.Meanwhile, the multi-threading computing speeds up the registration to realizereal-time performance.(5) After investigating the exisiting approaches of tracking and registration forAR, a novel tracking and registration method is proposed based on online learning ofnatural scenes. Feature points of natural scenes are effectively learned to improve thematching accuracy with the feature points of the captured image. The proposedmethod combines the learned features and the output of the optical flow featuretracking algorithm to achieve registration, which effectively improves the stability ofthe tracked pose and avoids the problem of wide-baseline matching. Meanwhile,real-time online learning ensures the precision and speed of tracking and registrationalgorithm.By studying the requirements of applying AR technology on outdoor complexenvironments, a mobile AR system is developed based on the algorithms proposed inthis paper for applications under outdoor environments.Experimental results in theapplications of campus guidance and museum navigation proves the performance ofthe proposed algorithms.