An AR System Based On Gesture Interaction

For augmented reality applications on smartphone platforms,the intuitive interaction of gestures can offer users a more natural operating experience.However,due to the hardware performance of the mobile phone and the restriction of the monocular RGB camera,there is currently no mature and complete solution to implement an AugmentedReality system with gesture interaction on the Android platform.An Augmented-Reality system based on gesture interaction on the Android platform is designed and implemented,and the two technical difficulties of hand pose estimation and dynamic gesture recognition are solved.Finally,combined with the gesture interaction module,the operation of controlling the movement and rotation of virtual objects in the scene is realized.As for the problem of hand pose estimation,a three-stage algorithm is designed and implemented.First hand detection and segmentation are performed,twodimensional key points are detected with the segmented image,and then the threedimensional position and posture of the hand are estimated according to the principle of camera perspective,also the rotation and translation relationship between the hand coordinate system and the camera coordinate system is determined.For the problem of dynamic gesture recognition,the traditional Dense Net is structurally improved,and a temporal transition layer composed of multiple convolution layer in parallel is introduced to capture information of different time lengths,instead of asymmetric convolution.The standard convolution to is replaced by asymmetric convolution to achieve compression and acceleration.Experiments show that both hand gesture tracking and dynamic gesture recognition algorithms can achieve high accuracy.Users can perform real-time gesture interaction through this system,and the system recognition speed and accuracy of this system can achieve the desired effect.