The Implementation Of AR Globe System Based On Convolutional Neural Networks

With the popularity of several AR products,augmented reality technology has aroused heated discussion among the public.Augmented reality(AR)is a real-time view that combines virtual information with the real world.It allows computer-generated perception information to ideally across multiple sensory modalities,including vision,auditory,olfactory,touch and somatosensory to enhance the elements of the real world.Since AR was proposed,it has been applied in various fields,especially in children education.This paper designs and implements an AR globe system for mobile smart phones based on convolutional neural networks that aims to help children learn knowledge of geography and history from the globe more efficiently.The AR globe system contains two core modules: national identification module and AR display module.In the national identification module,this paper chooses the most popular and the most efficient algorithm,convolutional neural networks technology,in the field of image recognition as the recognition algorithm,and proposes a network applied to the AR globe system based on the MobileNet.The improved operation is to group feature maps after depthwise separable convolution,which can not only combine features,but also reduce the training parameters and the complexity of the model.We perform comparison experiments between model proposed in this paper and several classic convolutional neural networks on the dataset we have built by ourselves.The results show that the network proposed in this paper achieves a high accuracy of 99.077%,which is second only to 99.599% of the ResNet.In addition,our model has the smallest size,only 0.352 M,and the minimum training parameters,a total of 87069.Another set of comparison experiment shows that the improved operation proposed in this paper does have a certain degree of optimization effect in terms of recognition accuracy,model size and training parameters.In the AR display module,in order to make up for the shortcomings of the current AR globe on the market,we use the three-dimensional registration method based on natural features in place of the method based on logos to make the system can be applied to a variety of teaching globes and has a certain degree of universality.The three-dimensional registration process firstly uses the ORB algorithm to perform feature detection and then matches the input image and the template image to obtain matching point pairs.Using these matching point pairs to calculate the homography matrix between the input image and the template image,and then calculates the external parameter matrix of the camera.Combining the external parameter matrix and internal parameter matrix of the camera to construct a projection model to determine the correct position of the virtual object in the real scene.Finally virtual-real fusion technology is used to render 3D virtual objects in the real scene.In the actual development process,we use OpenCV to implement 3D registration,3ds max to create 3D virtual models and Unity3 D to realize virtual-real fusion.In addition,we use Unity3 D as the integrated development environment to export Android and IOS installers.