Research On Lane-Keeping Control Using Safe Reinforcement Learning And Its Verification In SUMO

Autonomous driving can possibly facilitate the load and change the methods for transportation in our everyday life.Work is being done to create algorithms of decision making and motion control in Autonomous driving.Recently,reinforcement learning has been a predominant strategy applied for this purpose.But,problems of using reinforcement learning for autonomous driving is that the actions taken while exploration can be unsafe,and the convergence can be too slow.Therefore,before making an actual vehicle learn driving through reinforcement learning,there is an urgent need to solve the safety issue.The significance of this study is that,it introduces Safe Reinforcement Learning(SRL)into the field of autonomous driving.Safe reinforcement learning is the method of adding constraints to ensure the safe exploration.This study explores the Constrained Policy Optimization(CPO)algorithm.The principle is to introduce constraints in the cost function.CPO is based on the framework of the Actor-Critic algorithm where the space that is explored during the policy update process is enforced by setting tough constraints which reduces the size of policy update.The main work of this research includes the principle,theoretical proof and derivation of the CPO algorithm,and its actual implementation process,through the design of simulation experiments.There are various maps used to evaluate the performance of the algorithm.The security and stability of the CPO algorithm on different maps is verified.A comparison is also made with typical reinforcement learning algorithms to prove its advantages in learning efficiency and safety.This study is expected to provide the ground for the use of safe reinforcement learning in the field of autonomous driving.