Learning-based Hand Pose Image Style Transfer Method

Gestures have the characteristics of rich gestures,flexibility and convenience,and have a high research significance in the field of human-computer interaction.Accurate estimation of gestures depends not only on the design of the algorithm,but also on a large set of gesture dataset.The gesture dataset consists of two parts: the gesture depth image and the annotation of the joint points.The depth image can be acquired by the depth sensor or the data glove.The difficulty lies in how to accurately mark the hand joint points in the large-scale gesture image.The existing hand pose datasets have more or less problems of labeling and labeling errors,and the errors of labeling will inevitably affect the accuracy of the algorithm and the practical application value.In contrast,the 3D model data of the hand rendered by the software has accurate annotations,but the model data is difficult to use in practice.Aiming at the problem of existing dataset annotation,this paper proposes an image style conversion algorithm to transform the hand real depth image and model data.Specifically,this paper improves on the basis of CycleGAN's network,making it more suitable for gesture images with less color information and single background information.The real data and the model data are used as images of two domains.In the confrontation training,the generator is used to convert the image styles between different domains to generate a stylized image,and the discriminator is used to identify the generated stylized image and the real image,and feed the result back to the generator to promote it produces higher quality images.Cycle consistency is that the image of one domain is transferred to another domain,and then can be returned to the domain after a conversion.Adding a cycle consistency check in the network is to constrain the image content,avoiding the GAN mode collapse caused by the confrontation training.The real data is transformed into model data after the style conversion.The position of the joint point can be estimated by estimating the gesture posture network trained by the model data,which is used to update the original error label or generate the label;after the model data is transferred into real data expands the original gesture data set,and this part of the data is accurately labeled.In order to train the proposed network model,this paper generates a hand pose dataset through the model software,which contains 40,000 gesture images and three-dimensional annotation of hand joint points.Through a large number of experiments,this paper verifies that the proposed algorithm can correctly transform the real gesture data and model data and the effectiveness of each module in the network.Compared with other style conversion methods,the method has good performance.