Research On Ar Multi-person Collaborative Digital Twin Assembly Method Based On Scene Semantics

The assembly of large and complex products such as metallurgical equipment involves many parts and components,the assembly process is complex,and requires high precision assembly operation according to the process.The assembly of these core components of equipment relies on the collaboration of multiple people,which has become the key to the quality and efficiency of equipment assembly.However,traditional assembly relies heavily on the proficiency of employees and the tacit understanding between employees,resulting in low overall assembly efficiency and insufficient quality stability.In view of the urgent needs of engineering,this paper proposes an augmented reality multi-person collaborative digital twin assembly method based on scene semantics.The main contents include:Aiming at the problems of low information integration and poor scene adaptation of traditional assembly methods,the characteristics of multi-person collaborative assembly are analyzed,and a digital twin multi-person collaborative assembly method integrating AR is proposed for single-station and multi-station collaborative assembly.Considering the multi-source heterogeneous data of the AR collaborative process,the digital twin assembly information modeling is carried out based on the knowledge graph,and a process-oriented digital twin AR assembly information representation system is proposed.In order to solve the problems of inconsistent perspectives and difficult cooperative interaction of multiple people in AR assembly process,this paper proposes AR collaborative assembly methods based on digital twinning,including process collaboration,scene collaboration and data collaboration.Firstly,AR scene modeling is performed for the assembly process and process decomposition of the assembly sequence is performed.Secondly,taking the process as the basic unit,the transmission and iteration mechanism of collaborative process assembly information between the physical assembly scene and the digital twin space is established,which improves the adaptability of AR collaborative assembly.On this basis,the collaboration method based on root anchor point is adopted to realize the AR scene collaboration of multiperson assembly process,and rely on the real-time knowledge support provided by the assembly process knowledge graph to achieve data collaboration.In the process of data collaboration,precision monitoring is carried out through the assembly dimension chain and assembly error tracing is carried out according to the fluctuation coefficient,which improves the quality stability of collaborative assembly.Aiming at the lack of scene perception of the physical world in the AR collaborative assembly process,resulting in poor continuity of AR-assisted assembly and low fidelity of assembly virtual-real mapping,this paper proposes a collaborative task-driven method based on scene semantics.Firstly,based on object detection,sparse graph generation,and relationship prediction,the scene graph construction in the assembly environment is carried out.The generated scene graph is used for assembly task driving,and the abnormal perception and early warning are carried out by combining the graph node information and semantic relationship,so that the assembly process is implemented in accordance with the pre-planned process,and the standardization and accuracy of the assembly process are improved.Secondly,the assembly progress is perceived by the assembly sub-diagram,the frequent humancomputer interaction process is reduced,the continuity of the collaborative assembly process is improved,and a process recommendation method based on graph similarity matching is proposed to enhance the exception handling ability of the AR system.Finally,taking the rolling mill and gear pump assembly as the case background,this paper verifies the method from four aspects: process collaboration,scene collaboration,data collaboration and scene awareness,and compares the collaborative efficiency of the physical assembly process.The results show that the collaborative assembly method proposed in this paper can adapt to different assembly conditions,improve the assembly quality and assembly efficiency of complex products,and have certain application value.The AR multi-person collaborative assembly system based on digital twin developed in this paper has been put into operation by MCC company,which has good practical results.