Control Of The Inverted Pendulum Based On Reinforcement Learning

Reinforcement learning is different from supervised learning in that no teacher signals are needed. And reinforcement learning is different from unsupervised learning of few functions in that it obtains the rewards from the environment. Reinforcement learning system learns to optimize decision by the feedback information from its interacting environment. It adopts the mechanism of "action—critic —progress"that is used by people and animal in study. Reinforcement learning closely correlates with animal learning theory, cognitive science and autonomous learning machine. So reinforcement learning methods have wide application areas in solving complex optimization and decision problems where teacher signals are obtained hardly. In recent years, reinforcement learning is set store by domestic and overseas researchers in the artificial intelligence field. Because it is important for the study of autonomous robotic agents to research motor balance control, we emphasize to research cognitive models for motor skill. The paper mainly researches and improves the mature reinforcement learning methods. And the reinforcement learning methods proposed by the paper are applied to control the inverted pendulum system. The objective is that the intelligent systems can learn from run process and have motor skill similar to people and animal. The production can be applied in machine learning, automation and robotic fields so on. The paper had the main achievements as follows: (1)Based on Q learning algorithm storing information by table, the paper proposes the improved Q learning based on neural networks. It adopts neural networks to approximate Q value function and the SoftMax decision of Boltzman distribution to select the action. It can effectively control the task of continuous states and discrete actions. It is validated by the simulations of controlling the inverted pendulum system. (2)Based on reinforcement and dynamic programming, the paper proposes an improved reinforcement learning system using internally recurrent nets(RLSIRN). The learning system does not require a prediction model and an identification model. Even if the model of the system is not available and the leaner has no a priori experience, it can effectively control the task of continuous states and actions by adjusting itself online. The results demonstrate that it is superior to other congener...