Research On Robotic Visual Control With Deep Reinforcement Learning

As a core component of Intelligent Manufacturing and industry 4.0,robotic visual control combined with deep reinforcement learning method has draw much attention with recent development of deep learning in visual tasks and the application of reinforcement learning in sequential decision making problems.Research described in the paper focuses on deep reinforcement learning and mainly contains an attention-mechanism-based object detection algorithm,a traditional-control-based policy network architecture,a step-by-step robot control scheme and an end-to-end robot control scheme:1.Object detection algorithms generally search through extensive potential areas without considering spatial correlations.To fully utilize rich information contained in highlevel image features,a hierarchical object detection method with parallel search is presented.An attention-based state initialization algorithm combined with a novel reward function for RL training are proposed to facilitate the agent's control over window transformations.Compared with existing detection algorithms,experiments on PASCAL VOC 2007 & 2012 dataset indicate the proposed model achieves encouraging object detection performance with fewer proposals generated.2.Constructive reinforcement learning(RL)algorithms have been proposed to focus on the policy optimization process,while further research on different network architectures of the policy has not been fully explored.A "Proportional-Integral"(PI)neural network architecture that could be easily combined with popular optimization algorithms is proposed.Experimental results on public RL simulation platforms demonstrate the proposed architecture could achieve better performance than generally used MLP and other existing applied models.3.To deal with the dependency on system model and the lack of robustness in traditional control schemes,a step-by-step control scheme and an end-to-end control scheme based on deep reinforcement learning are proposed.Both schemes require visual input.The step-by-step control scheme combines the object detection network and the policy network after separately training,but the generalization is poor and the training process is much complex;the end-to-end control scheme only needs raw images to predict the best action in different control tasks.Experiments in the simulation environment prove that deep reinforcement learning enables the robot to learn manipulation skills from scratch.The proposed object detection algorithm as well as the PI policy network and end-to-end control scheme have great generalization.In addition to robot control tasks,they can also be applied to other reinforcement learning problems and visual tasks.