Application Of Deep Reinforcement Learning In Futures Trading Decision Making

The development of financial transaction strategy has always been an important In recent years,using deep reinforcement learning to make investment decisions is a hot research direction in the field of quantitative investment.This paper combines reinforcement learning and deep learning methods to build a futures trading model,in order to guide investors' investment practice.First of all,this paper describes the basis of model construction and combs relevant theories including reinforcement learning and deep learning.Then,the design of the key elements of the model,including state element design,action element design,reward value element design and the design of two different neural network structures,is explained.In terms of state element design,this paper innovatively proposes a market trend variable based on technical indicators and adds it to the model strategy.At the same time,compared with the model strategy without market trend variables,it is found that the introduction of market trend variables can improve the profitability of the model strategy.Therefore,in this paper,market trend variables are introduced into the model strategy,and two types of models with market trend variables are constructed,respectively,which are reinforcement learning models based on FC neural network structure and LSTM neural network structure.Secondly,this paper studies the trading performance of the two model strategies in different market conditions,and compares the profitability and risk resistance of the buy-hold strategy using the indicators such as annualized return rate and Sharpe ratio.The results show that,in the rising trend,the two models constructed in this paper can steadily and continuously obtain positive returns,but their overall performance is slightly worse than the buy-hold strategy.Under the fluctuating market trend,the two deep reinforcement learning model strategies constructed in this paper have achieved steady returns in three different stock index futures markets,and their ability to resist risks is far better than the buy-hold strategy.Finally,this paper summarizes the empirical results and finds that the deep reinforcement learning model strategy constructed in this paper is more suitable for volatile trading in the stock index futures market.However,the model strategies based on FC and LSTM network structures have different performance and robustness in different stock index futures markets.The model strategy based on FC performs better in IF and IC markets.The reinforcement learning model strategy based on LSTM has a better performance in the IH market.