Deep Reinforcement Learning Method Via Adaptive Weight And Double DQN

Deep reinforcement learning(DRL)received much attention within the artificial intelligence community.Recent results indicate that the experience replay strategies may greatly impact the learning performance in DRL.However,the appropriate selection of samples in deep Q-value network(DQN)and experience replay still remains an open problem.Firstly,to address this problem,this paper propose a new deep reinforcement learning method based on adaptive weighted Double DQN(DDQN),where an optimized DDQN designed with activation functions,optimization algorithms and loss functions is proposed to avoid the problem of excessive Q-value in DDQN.Secondly,the weight adaptive strategy is adopted in the experience replay to achieve balanced experience replay,and three weight impact factors are designed,including the network structure,the reward of the sample,and the usage times of the sample used to avoid over fitting and under fitting in the process of sampling.Thirdly,the proposed balanced experience replay strategy is applied to other neural networks based on the replay mechanism,including Dueling DQN and DDPG,etc,to adaptively select appropriate samples to improve the utilization efficiency of training samples.Finally,Simulated experiments are run on the Atari 2600 platform to test the experimental effect of proposed AWDRL in this paper,with comparison to such state-of-the-art methods as DQN,prioritized experience replay(PER),PER+,PER++and deep curriculum reinforcement learning(DCRL).The experimental resultsdemonstrate that the proposed AWDRL in this paper achieves better performance than the state-of-the-art methods.