Linear Quadratic Optimal Control Problem With Uncertain Coefficient

Linear quadratic optimal control(LQ for short)problem is a kind of special control problem in which the state equation is linear with respect to state variable and control variable,and the cost functional is quadratic.The linear quadratic optimal control prob-lem is also a very important category in the optimal control problem.Because of this,the linear quadratic optimal control problem develops continuously.At first,Bellman began to study the linear quadratic optimal problem of deterministic systems,and then Kalman solved this problem with linear state feedback in 1960.Since then,it has devel-oped into a stochastic model.As one of the cornerstones of optimal control theory,the maximum principle and dynamic programming were used by Kushner to study stochastic linear quadratic model with Ito process for the first time in 1962.At first glance,one might think that stochastic LQ problems are just some conventional extensions of deterministic correspondence problems.However,the existence of control in the diffusion term makes the stochastic LQ problem significantly different from the deterministic problem.Stochastic control theory has both broad application prospects and many challeng-ing research topics.LQ problems are a very important type of optimal control problems,because they can model many problems in the application.More importantly,many non-linear control problems can be reasonably approximated by them.On the other hand,the solution to the LQ problem has excellent properties due to its simple structure and excel-lent properties.The LQ problem is an important and challenging research topic.In recent years,the research on this kind of stochastic control problem and its application has also attracted more and more attention.Therefore,many basic problems are still open.Although the LQ problem has already had such fruitful results,the methods and forms still need to be improved,and there is still much space for research.Horizontally,if new variations are made to the LQ problem,there is still a need and space for research in new field;vertically,the research on the more stable results and practical applications of the LQ problem can still be studied continually.Generally speaking,new forms of LQ problems often need to analyze the solvability first,and then study the solution methods of the problem.And we often start with the deterministic or constant coefficient LQ problem,and then deeply explore the stochastic and variable coefficient LQ problem.In terms of applications,investors' consideration is to maximize the terminal average wealth EXT while minimizing the variance in terminal wealth.In the financial market,the traditional investment methods are mainly divided into two types,one kind of risk-free assets called bond,and the other is the risk assets called stock.The characteristics of bond are low risk,but the rate of return is also relatively low,while stock is the opposite,with high return but high risk.Therefore,in order to reduce risk and increase return at the same time,investors with different risk preferences can allocate assets to different markets according to their own proportions.In this regard,Zhou-Li abstracts the continuous-time mean-variance problem into a class of stochastic LQ problems,thereby obtaining a equilibrium strategy of investment in certain situations.However,in the LQ problem,when the coefficients of the state variable or control variable in the drift and diffusion terms are uncertain,for example,in the portfolio problem,the risk asset return rate is uncertain,and there are still few related studies.This Difficult new topics are different from the previous LQ problem,which requires the maximization of uncertainty risk.And this is also a problem that this thesis will discuss.In the era of rapid development of science and technology,at the same time,real life is also full of more and more uncertainties.It brings many disadvantages(and sometimes,surprisingly,advantages)to human efforts.For example,in the field of aircraft control,financial investment market,risk assessment and so on.There are higher requirements in the optimal control model now,so it also provides a certain direction for our further research.The uncertainty mentioned here not only has the randomness described by tra-ditional stochastic processes such as Brownian motion,but also includes the uncertainty that a certain parameter independent of Brownian motion can change in a certain interval.In many cases,this kind of uncertainty is neither random nor fuzzy,and it can not be de-scribed by mathematical language.However,sometimes a range can be found to cover it.Therefore,the uncertainty of the latter can also appear in the deterministic optimal control model.Information incompleteness is one of the reasons for this kind of uncertainty.In order to solve the LQ problem under the uncertainty coefficient,on the basis of the re-search of relevant scholars,considering the influence of interference and uncertainty,we hope to find a control which can make the performance index minimum,that is,robust optimal control,when the interference is as large as possible.The saddle point describes the optimal control in the worst case to some extent.However,as a minimax problem,it is difficult to solve these models directly,but they can be converted into their dual problems in the presence of saddle points.Therefore,we first do some research on the solvability of the deterministic LQ problem,and then try to find a way to solve the problem.Next,we further study the stochastic LQ problem with the uncertain coefficients driven by Brow-nian motion,and discuss the solvability conditions in this case,so as to further solve the problem.Then,the stochastic LQ problem is applied to the portfolio selection model with uncertain returns.From the point of view of the solution,if the minimax problem is solved directly from the obverse side,it will become a nonlinear control problem,which is more difficult.Therefore,this article considers trying to convert back to a linear problem and using a classic linear method to solve the model.The basic idea is to first express the model as a quadratic functional J(a,u)with only uncertain coefficients and control variable by using the theoretical method of functional analysis and backward stochastic differential equa-tion.According to the characteristics of this functional,the conditions for the existence of saddle point are found.In this case,we can convert the minimax problem into a maxmini problem,at which time we can use the classical Hamiltonian system method to solve.In this paper,from the perspective of calculus,functional analysis,stochastic calculus and stochastic control theory,the LQ problem with uncertain coefficients is discussed by means of model analysis,theoretical derivation and calculation and other methods in accordance with the principle of gradual progress from easy to difficult.In this paper,from the deterministic linear quadratic optimal control model to the stochastic linear quadratic optimal control model,and then to the portfolio model whose utility is quadratic function,we solve them one by one.Firstly,this paper studies the linear quadratic optimal control model with the given initial value of x,the state equation of dxt?(Axt+But)dt and the cost functional of J(A,u)=?0T1/2(Qxt2+Rut2+2Sxtut)dt.In this paper,it is assumed that A is the uncertainty coefficient.In this case,the theoretical method of functional analysis can be used to simplify the cost functional into a tractable bivariate quadratic functional form.Then,by using the linear and positive definite properties of the functional for u,we get the convexity of u,so we can get the conclusion that the saddle point exists when J(a,u)is concave.Next,we considers this model where there is saddle point that can convert the minimax problem to the maxmini problem.So,the optimal control u is solved by the traditional maximum principle.So we obtain the value functionJ(A,u),then maximize it with respect to A.Then further,this paper studies the stochastic model with Brownian motion.In this case,the state equation is dxt=(axt+ut)dt+cutdWt,where a is the uncertainty coeffi-cient and the cost functional is J(a,u)=E?0T1/2(xt2+rut2)dt.For this model,this article also considers some solvable situations step by step.We solve the state equation in the form of the control variable u and substituted into the cost functional.Then the cost func-tional can be represented as a quadratic functional of u with the parameter a by using the theoretical method of backward stochastic differential equation and functional analysis.Then,similar to the previous chapter,we find the conditions for the existence of sad-dle point.And then we still consider the existence of the saddle point where the minimax problem can be converted into the maxmini problem.We use the classical stochastic max-imum principle firstly to solve the optimal control u,and get the value function J(a,u),which depends on a only.We then maximize it about a.Subsequently,a special example without considering whether there is saddle point or not is given,and the relatively simple stochastic model is analyzed,discussed and solved.Finally,we consider applying it to an actual portfolio problem to illustrate the prac-tical significance of our result.To achieve this goal,we also only need to consider one-dimensional variables,so this article assumes that there is only one risk-free asset and one risk asset.After derivation and calculation,we build the model that the state equation is dxt=(axt+but)dt+cutdWt,and the cost functional is J(b,u)=E(?xT2-xT).In this model,b is the uncertainty coefficient.Different from the previous chapter,the un-certain coefficient of this chapter is on the control variable of the drift term.In order to solve the model,this paper still expresses the state variable as a function with parameters b and control variable u,and substitute it into the cost functional.Then use the optimized theoretical method to analyze and compare the influence of parameters on the optimal so-lution.Finally,try to solve the model where the cost functional has only terminal utility.