Research On Sequential Decision Problems Under Uncertainty

With the rapid development of computing, embedded technology, sensor technology, communication and automatic control technology, there comes into being intelligent system of the next generation. The intelligent system has extensive application field, such as smart city, intelligent transportation, military defense, health care and environmental monitoring, which has attract much attention from researchers. This article means to research sequential decision problem in Robot control and decision problem.Uncertainty is one of the main features of Robot control intelligent system. The complexity and uncertainty of the system have determined that node's (Robot) decision-making is meant to have problems of inconsistent and partly observed information, also problems of system distribution. How to make suitable decisions in appropriate time with limited information under such environment is node's primary problem in planning and decision-making. Markov decision theory has provided a solid mathematical foundation and model representation for decision-making under uncertainty.On the analysis of Markov decision research status, the paper proposes ESVI algorithm and IGA algorithm aiming to the disadvantages of Markov model's algorithms. In the first place, Evolution Strategy Based Value Iteration (ESVI for short) is proposed for POMDP model. ESVI selects optimal action under certain belief-state via constructing utility matrix based on random iteration process, after which adopts Bayesian rule to update belief-state. Random iteration process selects population using evolution strategy and update utility matrix due to the selected population. At the end of this chapter Tag problem and Hallway2 problem are solved using ESVI, experiments show that ESVI can obtain better profit value and approximate optimal action strategy when solving large size POMDP problems. In the second place, Improved Genetic Algorithm (IGA for short) is proposed. IGA definite optimal initial state and optimal profit state based on the analysis of states set, and divide the algorithm into two parts: strategy from initial state to optimal initial state and strategy that allow states transform between optimal profit states, which reduces the complexity of IGA. The purpose of the first part is guide decision-making node to optimal initial state, and this process actually wipes out invalid actions that exist in sequence of action. All the two parts of IGA is based on GA, but still have differences on genetic manipulation and the definition of fitness function. Grid meeting problem and Multi-access broadcast problem are solved in the end, experiments indicates that IGA compresses strategy space, reduces coding length, and IGA is an effective approximate algorithm to resolve DEC-POMDP problems.