The Application Of Value Of Information In Markov Decision Theory

In the past few decades,with the emergence of many new models involving uncertain factors and sequential decision-making problems in economic theory,communication engineering,business management and many other disciplines,the Markov Decision Process(MDP)modeling theory under the uncertainty has developed rapidly.In the field of artificial intelligence,MDP is a basic theoretical model for modeling and planning decision problems,and also a major research area of sequential decisionmaking.Since ancient times,information has been linked to the elimination of uncertainty.In the process of decision-making,the purpose of computing value of information(VOI)is to guide information collection process under uncertain environment,improve the quality of decision-making,and ultimately achieve the optimal decision.The main work is summarized as follows:(1)This paper presents a novel framework: Belief-based value of information decision model(Belief-VOI)aiming at a specific type of optimal uncertain sequential decision problems that need achieve the best trade-off between decision qualities and cost.It judges the optimal stopping time through the value of information in decision control,provides an analysis and solution tool for the balance between the immediate cost of collecting the information and the expected benefit of collecting more observations on future decision behaviors.(2)In addition,the proposed Belief-VOI theoretical model is applied to quality control in crowdsourcing task.It verifies the correctness of the theory and illustrates the scientific and practical significance of judging the value of information in decision analysis.(3)Because of the combinatorial challenge of the state space when calculating the optimal policy of the general Markov decision model,this paper considers the special structure of Belief-VOI model and considers a more efficient approximation method: A Monte-Carlo Tree method computing the value of information(BMCT)based on belief states.This simplified method is feasible and practical.This paper makes a deeper exploration of the application of the optimal uncertainty sequential decision problem and the value of information computation in the theory of Markov decision process.It achieves a breakthrough in the future of artificial intelligence technology in decision analysis.