Research On Obstacle Avoidance And Navigation Based On Evolutionary Computation And Deep Reinforcement Learning

With the rapid development of navigation technology,obstacle avoidance and navigation have played an increasingly important role in people's daily life and social activities.Traditional navigation methods are mostly realized through path planning in an environment where the map is known.When facing an uncertain environment with unknown maps and complex scenes,it is necessary to map the environment through simultaneous localization and mapping.These methods all rely on high-precision global maps,which lead to certain limitations in unknown map environments or complex environments.The agent realizes the strategy of direct mapping from state to action by interacting with the environment through deep reinforcement learning.With the help of deep learning to extract features from complex environments,real-time and end-to-end navigation control decisions can be realized.However,due to the complexity of the loss function and the existence of certain deceptive rewards,deep reinforcement learning may cause the agent to fall into a local optimal solution.At the same time,evolutionary computation,as a black box optimization algorithm without gradient information,can jump out of the local optimal solution and approximate the optimal strategy.This thesis mainly combines evolutionary computation and deep reinforcement learning algorithms to conduct research on the methods of navigation with obstacle avoidance.The main contents and contributions are as follows:(1)The navigation process is built as a Markov decision process based on the structure of deep reinforcement learning.The state and action space are established,which are corresponding to the obstacle avoidance and navigation task.In addition,in order to avoid the slow or even unconvergent learning process caused by sparse rewards,a dense reward function is designed.(2)Aiming at the navigation task of continuous control quantity,a navigation algorithm based on estimation of distribution algorithm and deep reinforcement learning is designed.The deep reinforcement learning algorithm is used to promote the convergence of estimation of distribution algorithm,and a large number of experience samples generated by estimation of distribution algorithm are used to train deep reinforcement learning algorithm to improve the sample efficiency.At the same time,the novelty search mechanism is added into the fitness function design of estimation of distribution algorithm to encourage agents to explore more and dynamically achieve the balance between exploration and utilization.The simulation experiment demonstrates that this method has a better learning effect than the other algorithms.(3)Considering the large variance of the obstacle avoidance and navigation combining distribution estimation and deep reinforcement learning,natural evolution strategies can be used to directly search for the natural gradient in the parameter space instead of estimation of distribution algorithm.At the same time,the search is assisted to accelerate the strategy search by the gradient information in deep reinforcement learning.In addition,natural evolution strategies based on mutation constraints can effectively reduce the impact of mutation intensity on the results Experimental data shows that this method maintains a considerable success rate,and the reward variance is better than the algorithm that combines estimation of distribution algorithm and deep reinforcement learning.