Research On The Trading Strategy Based On Deep Reinforcement Learning

Quantitative trading strategy consists of three parts: data feature extraction,algorithm construction and learning.In intraday trading,transaction costs and transaction data mining are key factors affecting the profitability of trading strategies.The classical trading strategy RRL can reduce transaction costs through action feedback,but RRL does not consider the feature extraction.Deep reinforcement learning as a way to solve these problems,uses neural networks,on the one hand,to express Markov Decision Process,and,on the other hand,to extract high-dimensional abstract features.The main problem is how to apply the deep reinforcement learning method to build and learn the trading strategy from three aspects of data state representation,feature extraction and strategy representation to make profits in intraday trading.To solve the problems above,this paper proposes a deep reinforcement learning based trading strategy,called DDRRL,on the basis of research on RRL and deep reinforcement learning.First,considering the single factor of price return as the state representation,MODRRL is built based on RRL using deep neural networks.Then,DDRRL adopts multi-factor state representation and double feature extraction networks based on MODRRL network,to further improve average profits.The experiments show that two trading strategies are effective on IF and IC datasets.The main points of this paper are as follows:1)Build the MODRRL trading strategy.The length of time window for state representation and different networks for feature extraction are key factors in mining data features.By comparing different lengths of time window and the performance of different networks to extract features,the length of time window is selected as 120 and the multi-layer fully-connected networks are used.Considering the gradient vanishment due to the intraday time steps,LSTM is used to build the decision-making network.The multiobjective learning method is adopted to reduce the average loss.The experiment shows that this method increases the average profits by about 0.4 point on IF.2)Build the DDRRL trading strategy.Based on the network structure of MODRRL,experimental analyses are carried out in three aspects: four kinds of multi-factor states representations,feature extraction of short-term price changes and different strategy representation methods.The state representation based on price return and volume is adopted,and the feature extraction network based on the price return forecasting is added.The experiment shows that the double feature extraction networks improve the average profits by about 0.6 point on IF.3)Implement MODRRL and DDRRL trading strategies based on TensorFlow.Dropout,faster optimizers and so on are further studied to improve generalization.The experiments show that the average return of DDRRL is 1 point on IF with transaction fee at 1.5 points per lot,and 2 points on IC with transaction fee at 2 points per lot.