An Interactive Method For Telepresence System Based On Stereo Vision And Gesture Recognition

In recent years,with the popularity of mobile devices and wireless networks,telepresence technology is widely used in video-conferencing,video chat,remote monitoring and other fields.A telepresence interactive system can usually be divided into three parts,namely,the local space,the remote space,and the wireless network connecting these two spaces.Among them,the local space has a computing server,remotely interactable devices,and a telepresence robot having image acquisition and transmission functions;in the remote space,there are a sensing device and a interactive device.The traditional methods of interacting with the telepresence system are keyboard-based interaction,mouse-based interaction,joystickbased interaction,interaction based on keys which are on the display screen,and multi-touch.Although these methods can achieve the purpose of interaction,they are not flexible and natural,and they cannot bring immersive experience to users.This paper presents a telepresence interaction method based on stereo vision and gesture recognition.The method makes the telepresence robot become a avatar,move in the local space,and obtain two live real-time video streams of the local space through a binocular camera;the video streams are transmitted to the remote space through the wireless network,and are displayed on the smart phone's split screen,that is,left and right sides of the mobile phone screen respectively display the images captured by the left and right cameras;users in the remote space can perceive local space in real time through VR glasses and smart phones,and get immersive experience;the somatosensory controller in the remote space acquires the user's gesture data and recognizes the user's gestures;according to the acquired gesture datas,the virtual hand is drawn respectively in the two-way video images displayed on the split screen;users can control the movement of the virtual hand by moving his own hands,and can also change the gesture of the virtual hand by changing his own hand gesture to achieve the purpose of remote interaction with the interactive devices in the local space.Compared with traditional methods,this method is more flexible and natural,and has broad application prospects.