Design And Implementation Of Online Remote Guidance System For Augmented Reality Based On Rigid Object 6D Pose Estimation

With the rapid development of technology,an increasing number of high-tech products have penetrated into people’s daily lives.In life,it is inevitable to encounter the unclear use method of equipment,equipment failure and other situations.However,it is often difficult for ordinary people to cope with all the situations they may encounter,and relevant experts are not guaranteed to be on call,so there is a huge market demand for remote guidance systems.And at present most remote guidance systems limited is associated with the two-dimensional plane,this way for some without the personnel of relevant experience is not very intuitive.There are also some remote guidance systems that incorporate augmented reality to good effect,but these systems often require expensive equipment to use.Therefore,this paper designs and implements an online remote coaching system that incorporates augmented reality technology and can be deployed for use on mobile.The system works through the use of augmented reality technology.The user can combine the object 3D model to elaborate the problem encountered.Experts can also conduct professional guidance to users through the combination of AR special effects and 3D models.The system enables remote guidance by visualizing objects and guided movements,greatly improving the efficiency of remote guidance.Augmented reality technology requires real-time acquisition of the postural relationship between a given object and the camera,so the application of augmented reality technology also requires the help of the rigid object 6D pose estimation algorithm based on the instance level.The rigid object 6D pose estimation algorithms are usually divided into two types of algorithms: location algorithms and tracking algorithms.Between them,the positioning algorithm has higher accuracy and slower inference speed,which is unacceptable for the real-time augmented reality technology.The tracking algorithm satisfies the real-time requirement,but the current instance-level 6D-based pose tracking algorithm basically uses RGB-D data as input,which is obviously extremely high limiting for the equipment used in the system.Therefore,a position-tracking master-slave network for augmented reality applications using RGB images as input is proposed in this paper.Therefore,this article proposes a master-slave network for locating-tracking applications in augmented reality using RGB images as input data,combining the idea of knowledge distillation.By distilling knowledge from the locating network,a smaller network is obtained for rigid object 6D pose tracking.To train and test the network,this article constructs an RGB image dataset consisting of four objects and tens of scenes.Based on Unity3 D,Visual Studio Code,Xcode and other platforms,the front-end,end-to-end communication and system functions are implemented in this paper.As tracking requires an initial pose and is a process of accumulating errors,this paper combines C++ and socket technology to build a server for placing the locating network model.The initial pose is given and the tracking error is corrected by transferring information from the user to the server.The experiments carried out demonstrate that the master-slave network proposed in this paper achieves high accuracy in rigid object 6D pose estimation.Furthermore,the tracking network runs at an average inference speed of 129 FPS on PC and 25 FPS on mobile devices,which fully meets the real-time requirements.