Design And Implementation Of Virtual Inspection System For Comprehensive Mining Work Surface Based On Kinect Gesture Interaction

With the continuous promotion of "Made in China 2025" and other strategies,virtual reality technology is constantly being applied to the field of digital mines.In order to meet the demand of safe,efficient and green mining of coal mines,intelligent inspection has become a hot topic in current research.However,at present,aesthetic and application have not been fully integrated in the research of mine system based on virtual reality.In the aspect of functional inspection,pure manual inspection has high labor intensity and low efficiency,and robot inspection lacks universality of application.In terms of system interaction,the operator cannot exert the maximum degree of freedom without the use of input devices.Aiming at the above problems,the virtual inspection system of fully mechanized working face based on Kinect gesture interaction is designed and implemented in this paper.On the basis of the development of virtual system with high simulation degree,the system control mode based on gesture is established to realize the equipment inspection function of fully mechanized working face in a more natural interactive way,so as to achieve the purpose of real-time interaction among "man-virtual-real" in the inspection process.The main research contents and achievements of this paper are as follows:(1)An intelligent inspection device operation method based on gesture recognition +VR interface is proposed.Combined with the trend and technology of mine intelligent development,the method realizes on-site manual inspection and remote intervention in the centralized control center to assist the inspection of fully mechanized mining face,which has rationality and applicability.(2)The virtual scene and UGUI interface of fully mechanized mining face with high imitation degree are established.Based on digital twin theory,virtual image of physical prototype and virtual scene of fully mechanized mining face are established.The model texture and rendering,illumination and occlusion removal,particle system and animation are studied,and the visual effect of fully mechanized mining face is simulated in virtual scene.VR scene development and UGUI design method of user interface are discussed.(3)A gesture interaction method for fully mechanized working face inspection based on Azure Kinect is developed.The key technologies such as the development of Kinect device,the acquisition and database establishment of human gesture skeleton,data processing and gesture recognition are studied.Completed the design and establishment of the patrol gesture set,and realized the natural interaction between the patrol personnel and the patrol equipment.The gesture interaction method can meet the inspection operation needs of on-site operators and remote intervention personnel in the centralized control center,and has strong applicability under different lighting and complex background conditions.(4)It realizes the data communication among "man-virtual-real" and completes the final fusion of the system.Inspection personnel can directly inspect the real prototype by collecting the Kinect sensor device and processing the data of the virtual system.The real prototype sends the sensing information back to the virtual system through the sensors and information channels arranged on the real prototype to drive the virtual model in real time.The technology of serial communication and database communication is studied,and the real-time interaction between inspection personnel,virtual system and real prototype is realized.The virtual inspection system for comprehensive mining work surface based on Kinect gesture interaction constructed in this paper provides reference for intellectualization and rationalization of safety inspection of fully mechanized mining face.