Research On Quantitative Trading Based On Deep Reinforcement Learning

The strategies of quantitative investment has become a hot topic in the field of reinforcement learning.Especially,it has wide application demand and academic research significance in the fields of stock,foreign exchange,and futures.This paper studies the quantitative investment strategy based on deep reinforcement learning.The quantitative investment strategy based on deep reinforcement learning can adjust itself at any time according to the changes of financial market,realize the perception of market changes and make decisions,and make the quantitative investment intelligent.The main contributions of this work include:(1)A new quantitative investment method based on deep reinforcement learning.To solve the problem of low efficiency and long training time in the policy optimization process,an improved scheme with “Clipping PPO” was proposed.Clipping PPO retains the advantages of efficient use of the training data,like the standard PPO algorithm,discards the value network of the standard PPO algorithm to avoid the difficulty of convergence of the value network.Experiments show that the Clipping PPO can improve the speed of policy optimization and the performance.(2)An improvement scheme of "Action shaping " is proposed in order to deal with the problem of "The effectiveness of strategy output action is related to position status".Action shaping fundamentally ensures the effective action of strategy output,and ensures that the action of strategy output can be truly reflected in the trading environment.Through the experimental verification,it is found that action shaping makes the policy optimization more stable and improves the profitability of the strategy.(3)Advantage functions shaping.For the problems with allocating the return to different trading actions,advantage function shaping was proposed.Through shaping different advantage functions,we can distribute the return to the actions of establish a position,maintain a position,or exit a position,realize the balance of different actions optimization,adjust the transaction frequency of strategies,verify the improvement effect of different advantage shaping through experiments,and further improve the income of strategies.(4)To further improve the sample efficiency,a path leading method was proposed.Path leading makes the strategy more reasonable,and improves the quality of the collected tracks.The experimental results show that the proposed method greatly improves the training efficiency and the performance of the strategy.This thesis applies the method of deep reinforcement learning to the field of financial investment,enriches the theoretical researches of deep learning and reinforcement learning,and also enriches the financial investment tools.The proposed methods significantly improves the efficiency of algorithm training,improves the performance of strategy,and has reference value for the application of reinforcement learning in the financial field.