| Many learning problems encountered in engineering applications involve the synthesis of an unknown real-valued function, given only a sequence of scalar, noise-corrupted observations. The methods and analysis presented in this thesis provide a new, nonparametric approach to such learning problems.; A new learning strategy is proposed that extends existing concepts from the areas of reinforcement learning and stochastic approximation. Traditionally, reinforcement learning methods are used to solve discounted, infinite-horizon, optimal control problems involving time-homogeneous, finite Markov chains, for which the transition probability matrix is unknown. The work presented here applies to infinite-dimensional problems, where the state and control spaces are continuous.; Specifically, a method is presented for learning in a function space setting by means of a stochastic approximation procedure. The learning process is driven by a sequence of pointwise observation data consisting of random sample points and associated noise-corrupted function values. This method relies on the generalization, in a spatially localized manner, of pointwise data via a suitable sequence of generalizing functions. The learning process has a Hilbert space interpretation in which the generalizing functions become the kernels of a sequence of compact integral operators. One consequence of this fact is that the learning process presented here is not addressed by previous works on Hilbert space stochastic approximation methods.; A principal contribution of this thesis is the development of a stochastic approximation theory that permits the use of compact operators in an infinite-dimensional Hilbert space. The objective is to solve an unknown equation defined on this space. A sequence of estimates is generated iteratively from residual errors of the equation. The residuals are projected through a compact "gain" operator and perturbed in a random fashion. Under suitable regularity conditions on the operators and random perturbations, the approximating sequence is shown to converge in both a mean-square and almost sure sense to the actual solution of the equation.; Simulation results illustrating the use of the proposed learning algorithm are presented for two example problems: (i) learning an unknown real-valued function and (ii) learning to optimally control an unknown dynamical system. |
