Studies On Generalized Learning Automata And Its Applications

Learning Automata is a reinforcement learning approach,and the ultimate goal of Learning Automata is to learn the sole optimal action from the action set so as to achieve higher reward compared with other actions via interacting with the environment in terms of a sequence of repetitive feedback cycles.It is simple but effective,having solid theoretical background,applicable to stochastic environment,and having on-line learning ability.And hence,a lot of efforts have been made to the research and the applications of the Learning Automata.However,with the new questions raised,such as multidimensional function optimization under complex stochastic environment,the limitations of the traditional Learning Automata theory have been exposed and confine the applications of the Learning Automata.This dissertation systematically extends the Finite Action set Learning Automata?FALA?and Continuous Action set Learning Automata?CALA?based on three basic elements of Learning Automata?i.e.,action set,responses set,and state?,and applies these results to applications.Thus,this dissertation mainly focuses on the followings.First,this dissertation investigates existing Learning Automata algorithms,and makes a thorough research on the basic theory of Learning Automata,including the definition,structure and evaluation criteria.Second,this dissertation extends the action set and the state of the FALA under stationary stochastic environment.On the one hand,in order to overcome the limitaions of the tradional Maximum Likelihood Estimate?MLE?based Learning Automata?e.g.,cold start problem?,a new bayesian estimator is presented and proved to be the Laplacian smoothed value of the MLE.Based on the proposed bayesian estimator and stochastic estimator,the Generalized Bayesian Stochastic Estimator?GBSE?LA is proposed and proved to be?-optimal,which solves the cold start problem effectilvely and converges with faster speed.On the other hand,in order to overcome the limitations of the P-model stochastic environment,this dissertation introduces triple level stochastic environment,and proposes a general framework which extends the P-model Learning Automata to interact with the triple level environment.We also propose three learning algorithms with this framework to interact with triple level environment,TI-TFSLA_?2N,N??TI-IJA_?2N,N??TI-GBSE based on TFSLA_?2N,N??IJA_?2N,N?and the proposed GBSE,respectively.Third,this dissertation extends the responses set and the action set of the CALA under the stationary and non-stationary environment.In order to overcome the limitations of current research on SPL?i.e.,stochastic environment guides the learning mechanism with only two possible directions,“right”or“left”?,a new triple level environment is proposed.Random Walk based Triple Level Learning Algorithm is introduced to interact with this triple level environment and also proved to be?-optimal.In the proposed method,RWTA,the learning mechanism first generates a guess of the unknown point,which is achieved by subdividing the search interval into N steps,where N is the resolution parameter,and then performs a controlled random walk on this space.Theoritical and experimental results show that the RWTA solves the triple level problem under both stationary and non-stationary environment effectively when the convergence conditions are satisified.For another,we formulate the multi-dimensional Stochastic Point Location?SPL?problem,and build a new framework successfully soving this problem via decomposing the reliance amond paraments and transforming the multi-dimensional SPL problem into searching d optinal points on d different super lines,where d is the dimension size.Theoritical and experimental results also demonstrate that this framework is able to learn the optimal point when the convergence conditions are satisified.Finally,we apply the above algorithms to solving the practical problems.In order to evaluate and track the spatiotemporal events under stochastic environment,this paper proposes two LA based methods,STP-TFSLA and STP-GBSE,which transforms the current observation into the reward or penalty response,and uses the?-optimal LA to interact with this environment so as to suppress notifications if a regular pattern actually exists and notify if a hypothesized pattern does not exist even under the environment which might mislead the learning process with high possibility.As for the function optimization under stochastic environment,we also give the CALA based methods,MSPL-SUFO and MSPL-MFO both of which use the Random Search Based Algorithms Merged with multi-dimensional Stochastic Point Location?SPL?framework to optimize the multi-dimensional strict unimodal function and multimodal function under the stochastic environment which is composed of the random noise corrupted function values.