Research Of Mobile Augmented Reality Based On Natural Features And Sensor Information

Augmented reality enhance the user’s interactive experience with the environment by add digital information to the real world. In contrast to virtual reality technology, augmented reality virtual information will be more inclined to bring the virtual information into the real world, allowing users to perceive the computer-generated virtual models in real scenes. As one of the development trend of human-computer interaction, augmented reality has played an important role in medicine, machinery, entertainment, and some other fields. What’s more, it has considerable development prospects.In recent years, the rapid increase of smart phone’s hardware conditions has provided an important foundation for the development of mobile augmented reality. Smart phones have small size, light mass and are portable enough for our daily life. Compared with the personal computer, smart phones are more suitable to act as a medium between people and the world. However, the computing capacity of mobile devices typically can’t support real-time registration of augmented reality system. Therefore we can’t just transplant traditional3D registration algorithm to the mobile platform directly. To solve this problem, the paper focuses mainly on real-time registration of a mobile augmented reality system which is based on the natural features. The work can be divided into three parts:Since mobile devices have limited resource and weak computing performance, this paper improved the traditional feature description algorithm-FREAK, and proposed a gravity-based FREAK algorithm. In order to make key points have rotational invariance, FREAK assigns a main direction to the feature by analyzing the intensity of the key point’s neighborhood. While such kind of methods has been widely used among various feature description algorithms, it may occupy a lot of system resource and restrict the real-time efficiency of the algorithm, especially when the number of feature points is large. In this paper, we read current gravity data from the device’s gravity sensor and projected it to the image plane, then designated the gravity vector projection to be the feature point’s main direction. Compared with traditional method, gravity-based method eliminates the gray gradient calculation of each feature point and improves the efficiency of the algorithm finally. Since the gravity of real world object is vertical downward, this method can keep the rotational invariance of feature points. Meanwhile, the point’s main direction is more stable than before, as its accuracy is ensured by the device’s gravity sensor. Unfortunately, when the mobile phone is parallel or nearly parallel with a horizontal plane, the gravity can’t form a projection in the image plane. To solve this, we make a compensate plan in the fifth chapter.Traditional augmented reality systems on PC can achieve registration by using natural feature matching only. But this way is not available on the mobile platform, because it has to extract the feature point of each frame, which is unrealistic for a resource limited device. Take the video’s continuity into account, some researchers use KL algorithm to track the target scene in order to implementation real-time registration. However, using the KL algorithm will bring many restrictions, for example, it has a large chance to loss the target when the camera rotates rapidly. To solve this, we use the smart phone’s inertial sensor to provide initial parameters for the KL algorithm in every iterative loop, which increases the probability of convergence.Finally, based on the improvements above, we design and implementation a mobile augmented reality prototype system. The running result proves that our improvements are effective.