Research On 3D Gesture Estimation Method Based On CVAE-CGAN Model

Gesture estimation is an important research direction in the field of computer graphics and human-computer interaction.With the rapid advancement of computer computing speed and the reduction of hardware costs,gesture estimation based on deep learning has also developed by leaps and bounds.Gesture estimation has a wide range of applications in many practical scenarios such as autonomous driving,virtual reality(AR),augmented reality(VR),and smart home.Therefore,gesture estimation has become an important research field in the intelligent development of human society.However,due to the rich changes in hand posture and complex background of gesture recognition,the current gesture estimation still has many problems in recognition accuracy and real-time recognition.First of all,this paper analyzes and studies the existing methods of hand segmentation in gesture estimation.Through research,it is found that improving the accuracy of hand segmentation can improve the effect of gesture estimation.However,the traditional hand segmentation method based on semantic segmentation lacks detailed description.Therefore,this paper proposes hand segmentation based on fusion structure FCRN network.method.First,the RGB image in the gesture data set is processed by the NIN model to obtain a corresponding skin segmentation image;then,the RGB image in the gesture data set and the skin segmentation map obtained through the NIN model are input to the full convolution with the Atrous spatial pyramid pool.The residual network FCRN network performs deep fusion training.The Atrous spatial pyramid pool is used to optimize the feature extraction of the model.Finally,the experimental results show that the proposed method is superior to the traditional hand segmentation method.Secondly,this paper studies and analyzes the three-dimensional joint point detection method in gesture estimation.By analyzing the principle of variable-point automatic encoder(VAE)and the feasibility of cross-modal method in gesture estimation,a cross-modal gesture estimation method based on CVAE-CGAN model is proposed.The method adds the segmented hand information to the training of the VAE model,and makes the input information of the model training richer.At this time,the VAE model is transformed into the CVAE model;then,the alignment network CGAN is added in the hidden space of the CVAE.This method maps the hidden space of RGB image modality to the hidden space of 3D gesture mode,which is more accurate than the existing method of sharing hidden space with cross-modality.Finally,through experiments,the CVAE-CGAN model based on this paper is proposed.The cross-modal gesture estimation method is superior to the existing method of estimating 3D gestures through RGB images.