Research And Implementation Of Maintenance Support System For TBM Based On Augmented Reality

With the continuous advancement of technology,large and complex mechanical-electronic integrated equipments,such as tunnel boring machine(TBM),are increasingly used in social production.Complex equipment with technology-intensive and functional compound features can improve the degree of automation and efficiency of the operation,but it has presented difficulties to the technicians' maintenance work.The presenting issues of TBM manifest as the complex maintenance,which means technicians are required to use traditional maintenance support methods with negatively affect the efficiency of maintenance operations and operational safety hazards.This paper aimed to provide some solution based on augmented reality(AR)technology for the presenting issue,which lower maintenance difficulty and enhance the efficiency by enriching the maintenance guidance for technicians as well as increasing their sensory-ability through a virtual-real fusion maintenance environment.In this paper,we first combined the requirements of TBM maintenance and AR technology to analyze the requirements,and the architecture design of TBM maintenance support system based on augmented reality and the design of system functional modules inspired by the analysis of key support technology.Secondly,this paper proposed a system tracking and registration scheme based on natural features using ORB algorithm to extract and match the natural features of the characteristics of TBM repair scene.The matching results were optimized by introducing an improved RANSAC method.The real-time performance of the system operation was elevated by the KLT algorithm's predictive and tracking abilities.What's more,the homography projection matrix was updated using the feature matching results,and the virtual reality fusion effect between the virtual information and the real TBM maintenance scene was realized.Thirdly,a human-machine interaction method—based on hand-gestures was proposed building on the previous analysis of the relationship between maintenance status and behavior in the augmented reality maintenance support process.The depth image information was used to achieve gesture segmentation and contour extraction,and the gesture shape and position information are used as the basis for the rejection of invalid hand-gestures.The feature description vector based on image attributes was established,and the gesture classifier was trained by a support vector machine to obtain a system interaction gesture recognition model.Then,implementation method of the key technology of the prototype system was introduced,and the development of the augmented reality maintenance support prototype system was completed by using hardware and software platform tools such as Visual Studio,Unity3 D and AR smart glasses.The application verification was carried out at the TBM repair site.Finally,this paper summarized the primary work of the current study and discussed the limitations as well as given future research directions for other researchers.