Research On Portfolio Strategy Based On Deep Reinforcement Learning

Portfolio strategy is a subject that has been thoroughly studied in the financial field.However,various models based on certain assumptions are often difficult to be realized in the actual stock market due to their complexity and high cost.Deep Reinforcement Learning has many advantages and has been able to outperform human players in many challenging games.Based on the Deep Reinforcement Learning algorithm,this thesis studies the portfolio strategy to achieve better returns under the premise of controlling risks.First,this thesis defines the Markov Decision Process model of stock portfolio trading.Deep Deterministic Policy Gradient algorithm is applied to the model.In order to spread the weight of investment and optimize the portfolio strategy,the entropy of the portfolio is introduced as a regular term.A algorithm is designed,which takes price,stock share and fund balance as input,and outputs portfolio value.Compared with DDPG strategies studied in the past,the improved strategy has a significant increase in revenue.In another experiment,30 constituent stocks of CSI 300 Index are selected as the stocks traded in the portfolio.The experiment proves that this strategy has a higher return rate than the CSI 300 Index.But there is still possibility for improvement in trend tracking.Then,aim to improve the performance of the strategy to track the trend and identify the buy and sell signals,a portfolio strategy based on technical indicators is proposed.This strategy is based on Deep Reinforcement Learning.To further improve the return rate,this thesis takes both technical indicators and the covariance of portfolio stocks into consideration.The closing price,opening price,high and low price are used to calculate the technical indicators,And then calculate the covariance of the portfolio.The closing price,technical index,covariance,stock share and fund balance are input as the status,and the output is the portfolio value.Advantage Actor Critic algorithm and Proximal Policy Optimization algorithm are used to train the agents,which are more stable and efficient.Three experimental groups are set in the experiment for comparative analysis.The strategy proposed in this thesis has made significant progress than the CSI 300 index,Guangfa China Medical Index Fund and Guangfa Diversified Emerging Stock Fund in terms of return rate and Sharpe ratio.Finally,this thesis proposes an integrated strategy that uses Deep Reinforcement Learning to learn stock trading strategies by maximizing investment returns.To obtain an overall trading strategy,three actor-critic algorithms are used to train a Deep Reinforcement Learning agent.The three algorithms are Advantage Actor Critic,Proximal Policy Optimization and Deep Deterministic Policy Gradient.The integrated strategy inherits and integrates the best functions of the three algorithms so that it can robustly adapt to various market conditions.The integrated strategy has significantly improvement than the CSI 300 Index.The annualized return rate of the integrated strategy is 98.07%.The cumulative rate of return is 196.13%.The Sharpe ratio is 2.66.It proves that the strategy proposed in this thesis can be effectively applied to A-share market of China.