Research On Algorithms Of Free-of-parameter-tuning Learning Automata

Learning automata,an autonomous system optimizing autogenic decision-making mechanism by interacting with the environment,belongs to reinforcement learning schemes in machine leaning.It has attracted much research attention owing to its fast convergence,global optimal,strong robustness and complete theory.Also,it has been preliminarily applied in diverse areas,including pattern recognition,function optimization,path planning,etc.However,learning automata algorithms are apparently restricted by hyperparameter setting.In order to obtain the appropriate value of the hyperparameter,it is necessary to search the optimal parameter,and the parameter adjustment usually brings a lot of computational overhead.In particular,in interaction-costly scenarios,hyperparameter adjustment may bring high or even devastating losses,thus becoming a major bottleneck in the development of learning automata.Therefore,expanding learning automata theory form a hyperparameter perspective to fit it applicably in new scenarios becomes a growing research tendency in recent decades.Considering the drawbacks of hyperparameter searching,this dissertation studies parameter-free learning automata mechanisms,and expands the learning automata theory,focusing on finite and continuous actions,stationary and nonstationary environments.Below lists the precise contributions:In the first place,focusing on finite action set learning automata(FALA)theory,this dissertation analyzes the dependence of on parameters and the cost of finding parameters in terms of the most hyperparametric algorithms,and the limitation that can't get rid of the probability vector in terms of the only non-hyperparametric algorithm.A parameter-free tuning idea is designed to make the sampling strategy and termination condition independent of the probability vector.Thereby,the parameter-free FALA schemes from the perspective of loss function and estimation interval are respectively proposed.From the loss function perspective,this dissertation devises FALA algorithms with two actions and multiple actions,respectively,with rigorous convergence proofs.From the estimation interval perspective,this dissertation proposes FALA algorithms based on either confidence interval,in Frequentists,or credible interval,in Bayesians.Comprehensive experiments illustrate the effectiveness and performance advancement with computational cost decline of proposed algorithms.Secondly,focusing on continuous action set learning automata(CALA)theory,this dissertation analyzes the sensitivity to initial parameter settings and noisy environment settings in terms of the most existing algorithms.A parameter-free tuning idea is designed to update the probability density function and map the function to the environment.Thereby,the parameter-free CALA schemes from the perspective of insensitive parameter and non-parametric are respectively proposed.From the insensitive parameter perspective,this dissertation devises CALA algorithms by the indirect update of probability density function and the sigmoid function normalization,with rigorous convergence proofs.From the non-parametric perspective,this dissertation proposes CALA algorithms by the direct update of probability density function and the dynamic set normalization,and furthermore raises the precision-promoting algorithms either with a hierarchical or an adaptive structure.Comprehensive experiments illustrate the effectiveness and performance advancement with computational cost decline of proposed algorithms.Next,focusing on learning automata theory in non-stationary environments,this dissertation analyzes the performance impact brought by window function settings in terms of the most existing algorithms.A parameter-free tuning idea is designed to analyze based on the existing results and expand based on the aforementioned results.Thereby,the parameter-free schemes from the perspective of dimensionality reduction,mutation detection and time restriction are respectively proposed.From the dimensionality reduction,this dissertation devises a parameter default algorithm based on the classical algorithm,with rigorous convergence proofs.From the mutation detection perspective,this dissertation proposes an extended algorithm based on the aforementioned parameter-free tuning FALA algorithms.From the time restriction perspective,this dissertation proposes an extended algorithm based on the aforementioned parameter-free tuning CALA algorithms.Comprehensive experiments illustrate the effectiveness and performance advancement with computational cost decline of proposed algorithms.In summary,this dissertation studies in-depth the parameter-free learning automata mechanism,with respect to finite and continuous action sets,stationary and non-stationary environments,and proposes novel parameter-free learning automata algorithms,providing theoretical basis and experiment references for extensive learning automata-based solutions in practical scenarios.