| A conventional approach to estimation problems is the so-called Bayesian inference, in which the estimator employs an optimum strategy with respect to a pre-assumed probabilistic model on the data. Although the Bayesian approach has resulted in many efficient and practical estimation schemes, one of its main drawbacks is the fact that it assumes a pre-specified prior model on the data, which does not always correspond with practical scenarios, such as image denoising and target tracking, that give rise to model uncertainty and mismatches.;In this thesis, inspired by much of the work in information theory, we present an alternative universality approach that alleviates this drawback by constructing schemes that have guaranteed performances regardless of the statistical characteristics of the data. In particular, we focus on the case of noisy data, emanating from an unknown source corrupted by noise whose statistical characteristics are known to various degrees. We consider two types of estimators: the causal estimator and the non-causal estimator. The first is also referred to as a filter, and the latter a denoiser. For several different scenarios, we devise universal estimation schemes that achieve the performance of the optimum scheme that would have been designed with complete knowledge of the source and noise statistics. Our results demonstrate that as the data observation length increases, knowledge of the noisy channel suffices to learn about the source well enough to optimally estimate it from the noisy data.;More specifically, we consider three different problem settings. First, we devise a universal filter that attains the performance of the optimum filter for any stationary and ergodic, finite-alphabet source data corrupted by discrete memoryless channels. Next, we again consider a universal filtering problem with real-valued source and noise. With known noise variance, we devise a filter that universally attains the performance of the best FIR filter for any bounded source data, under the mean-squared error (MSE) criterion. Finally, we consider the problem of discrete denoising and generalize the recently introduced Discrete Universal DEnoiser (DUDE) to obtain a practical scheme that can compete with the best switching between sliding window denoisers. |
