Stochastic approximation algorithms with applications to particle swarm optimization, adaptive optimization, and consensus

n this dissertation, we present three problems arising in recent applications of stochastic approximation methods. In Chapter 2, we use stochastic approximation to analyze Particle Swarm Optimization (PSO) algorithm. We introduce four coefficients and rewrite the PSO procedure as a stochastic approximation type iterative algorithm. Then we analyze its convergence using weak convergence method. It is proved that a suitably scaled sequence of swarms converge to the solution of an ordinary differential equation. We also establish certain stability results. Moreover, convergence rates are ascertained by using weak convergence method. A centered and scaled sequence of the estimation errors is shown to have a diffusion limit. In Chapter 3, we study a class of stochastic approximation algorithms with regime switching that is modulated by a discrete Markov chain having countable state spaces and two-time-scale structures. In the algorithm, the increments of a sequence of occupation measures are updated using constant step size. It is demonstrated that least squares estimations from the tracking errors can be developed. Under the assumption that the adaptation rates are of the same order of magnitude as that of times-different parameter, it is proven that the continuous-time interpolation from the iterates converges weakly to some system of ordinary differential equations (ODEs) with regime switching, and that a suitably scaled sequence of the tracking errors converges to a system of switching diffusion. This work is an extension of the work in [80]. In Chapter 4, we developed asynchronous stochastic approximation (SA) algorithms for networked systems with multi-agents and regime-switching topologies to achieve consensus control. There are several distinct features of the algorithms. (1) In contrast to the most existing consensus algorithms, the participating agents compute and communicate in an asynchronous fashion without using a global clock. (2) The agents compute and communicate at random times. (3) The regime-switching process is modeled as a discrete-time Markov chain with a finite state space. (4) The functions involved are allowed to vary with respect to time hence nonstationarity can be handled. (5) Multi-scale formulation enriches the applicability of the algorithms. In the setup, the switching process contains a rate parameter...