Research And Application Of Key Technologies Of Human-computer Interaction And Equipment Remote Control In Virtual Reality Environment

In the research and development practice of intelligent manufacturing system,there will be a lot of human-machine interaction and teleportation of machine equipment.Many digital and intelligent factories have advanced human-computer interaction system and the ability of remote state detection,debugging and maintenance of equipment.Especially in high temperature,toxic,high ionizing radiation and other harsh conditions,engineering and technical personnel can remote control the field equipment through advanced human-computer interaction and remote operation technology,so as to protect the body of staff from damage.Human computer interaction and remote control technology have a large number of market demand and broad application prospects,the related research is very popular.With the development of virtual reality and 5g technology,new technology elements are injected into human-computer interaction and remote control technology.Human computer interaction and remote control based on traditional technology can not meet people's needs.People need to build a real-life visualization system,which can interact with the virtual scene naturally and control the equipment remotely in the virtual reality environment.Aiming at the development of human-computer interaction and equipment remote control system in virtual reality environment,this paper studies several technical problems needed to be solved in this system,including data acquisition technology based on vision,scene pre modeling and 3D display technology,low delay and safe remote control technology.The main contents of this paper are as follows:(1)In this paper,a method of collecting action state information of physical equipment by machine vision technology is proposed,and a variety of visual markers are designed to mark physical equipment.The depth learning algorithm is applied to the recognition and tracking of visual markers.The experiment shows that the introduction of depth learning algorithm simplifies the camera calibration process of binocular and multi eye vision,especially the need for multi camera,When the field of view of the camera overlaps irregularly,the method proposed in this paper can recognize the visual signs attached to the equipment and the human hand and realize the positioning of the visual signs,so as to indirectly obtain the attitude,action and other information of the attached objects.(2)In order to solve the problem of real-time construction of virtual reality scene,this paper designs a set of scene construction method that pre modeling of physical equipment and real-time loading of model with visual information.Different from 3D reconstruction technology,this method does not need complicated visual and laser scanning equipment and high configuration computer system.In this paper,we use the lens VR display technology to project the real-time scene to the left eye and the right eye respectively to realize the immersive stereo display.(3)Aiming at the problem of delay in control signal transmission and processing,this paper optimizes the communication link and proposes a method of model predictive display to reduce the delay of signal transmission.The remote control based on PLC is realized.(4)Based on HTV vive helmet display system platform,a set of human-computer interaction and equipment remote control system in virtual reality environment is developed and implemented.The system can construct 3D scene in real time according to the information collected by vision,and associate the operation in virtual environment with the entity device,so as to realize the real-time synchronization between the entity device and the virtual scene.