Research On 3D Planar Detection Algorithm For Mobile Augmented Reality

With the rapid development of mobile devices such as smart phones,augmented reality technology based on mobile terminals has become a hot research issue in the field of augmented reality.3D plane detection has attracted widespread attention as its key technology.Due to the limited computing performance of the mobile terminal,its plane detection is mostly obtained by texture feature matching,but it cannot detect weakly textured planes,and using deep learning to detect planes itself has certain challenges.Therefore,the dissertation has carried out research on the 3D plane detection algorithm for the mobile terminal,selected the U-Net network to realize the3 D plane detection,and designed a lightweight design for its network.After that,completed the migration and quantification from the computer to the mobile,and accelerated the hardware.The dissertation also uses the mobile terminal inertial measurement unit(Inertial Measurement Unit,IMU)to assist the vision to realize the real-time 3D plane detection of the mobile terminal.The main contributions are as follows:(1)Based on the analysis of a representative 3D plane detection network,a 3D plane detection method based on lightweight U-Net network is proposed.By analyzing the existing 3D plane detection network,the U-Net network is selected to realize the 3D plane detection.Due to the limited computing performance of the mobile terminal,and the deep learning network model has redundancy,the operation speed is low.In order to optimize the network structure and improve the calculation efficiency,a lightweight method for 3D plane detection network is proposed,including modifying the model convolution method,reducing the feature map resolution and the number of network channels.Experiments show that the network can reduce the redundant nodes of the network,reduce the amount of parameters and calculations,and improve the detection rate.(2)With the help of Android mobile deep learning framework,the dissertation has designed the model migration,quantification and hardware acceleration scheme of 3D plane detection network from the computer to the mobile.After the network is lightened,the model needs to be migrated so that it can be loaded and called on the mobile terminal.In order to detect the plane more efficiently,the dissertation uses a quantitative tool to quantify the model with INT8,reducing the amount of model parameters andreducing the size of the model.In addition,the dissertation also uses the mobile terminal hardware acceleration engine,including neural network application programming interface(Neural Networks Application Programming Interface,NNAPI)and Hexagon to complete the model acceleration.Experiments show that the proposed scheme can improve the detection rate.(3)According to the characteristics of multiple sensors in mobile terminal,a real-time 3D plane detection method based on IMU assisted vision is proposed.Because the calculation speed of 3D plane detection deep network is difficult to meet the real-time requirements of mobile augmented reality applications,the dissertation uses the mobile terminal IMU data assisted depth learning network to complete the real-time3 D plane detection.The method estimates the incremental movement of the camera in space by fusing IMU data,determines the trajectory of the camera in time and space,and assists in the detection of three-dimensional planes based on vision.