Research On Robotic Arm Training Method Based On World Model’s Hidden Variables And Dual-View Fusion

With the introduction of a series of national policies related to intelligent manufacturing,various styles of robotic arms have spread to various fields of human production and life,but the intelligence level of robotic arms is generally not high today.Unlike traditional robotic arm control,deep reinforcement learning algorithms do not need to build an accurate dynamics model,but train robotic arms through a large amount of data and trial and error to make them learn and adapt to the uncertainty in the actual environment autonomously.The world model is one of the most popular algorithms in the category of model based deep reinforcement learning,which can be used to solve complex decision making tasks by representing the environment in time and space,constructing a corresponding spatial model in time,and making planning and prediction based on the constructed model.However,the algorithm has less relevant application research in3 D robotic arm scenarios.If the algorithm is directly applied to do training in 3D robotic arm scenarios,it will lead to poor experimental results.In addition,the self-encoder in the Dreamer algorithm has a poor effect on the image reconstruction of the robotic arm scene,which will lead to a large variance of the return value in the experimental results and a low robotic arm grasping rate.In response to the above problems,this paper conducts an in-depth study.The main work of this paper can be divided into the following three parts:(1)for the shortcomings of Dreamer algorithm can’t be effectively used for 3D robotic arm scene,this paper improves the input side of Dreamer algorithm and proposes a Dreamer algorithm based on dual-view fusion(Dreamer-DVF).The algorithm inputs images of front and side view simultaneously,and the two images are encoded to get their hidden variables,and then they are fused with the hidden variables of the world model in turn.This algorithm improves the performance of the robotic arm trained on different tasks by enabling the world model hidden variables to learn and represent the 3D scene information as much as possible through the input of two view images.(2)To address the drawback of low accuracy of image reconstruction in Dreamer algorithm,this paper selects the VQ-VAE algorithm with the best reconstruction effect by comparing the performance of the selfencoder in Dreamer algorithm and six other VAE algorithm on image reconstruction.And the VQ-VAE algorithm is improved,and the one-hot extended VQ-VAE algorithm(One-Hot Extended Vector Quantized Variational Autoencoder,OHE-VQ-VAE)is proposed.The new algorithm extends its encoded hidden variables with one-hot encoding for replacing the autoencoder in Dreamer.The experimental results show that the Dreamer-DVF algorithm using OHE-VQ-VAE has less variance in the return value during the training of the robotic arm compared with the self-encoder,which indirectly improves the grasping rate of the robotic arm on the grasping task.(3)In order to verify that the improved algorithm also has better performance results in the physical robotic arm scenario,the experiments were migrated from the simulation environment to the real robotic arm scenario.The grasp rate of the Dreamer-DVF algorithm using OHE-VQ-VAE in the real robotic arm scenario when performing the grasp task under certain constraints and without pre-training was as high as 94.33%.