Partial likelihood theory and its application to communications

In this dissertation, we show how partial likelihood (PL) theory can be used to construct a flexible framework for development and study of signal processing solutions within a likelihood framework. Posing the problem in a likelihood setting, provides a number of advantages, such as allowing the use of powerful tools in statistics and easy incorporation of model order/complexity selection into the problem by use of appropriate information theoretic criteria. PL theory, we show, can be used to establish the fundamental information theoretic connection, to show the equivalence of likelihood maximization and relative entropy minimization, without making the assumption of independent observations, an unrealistic assumption for most signal processing applications. We show that this equivalence is true for the basic class of probability models, the exponential family, that includes many important structures that can be used as nonlinear filters. Examples are given to illustrate the application of PL theory to probability distribution estimation of continuous and discrete random variables.; When applying the partial likelihood formulation to multi-class posterior distribution estimation, we note the inefficiency of training the softmax model and propose an effective multi-classifier structure based on binary coding of the output classes. When estimating the class conditional densities using PL theory, we construct the information geometry of maximum PL estimation and derive the sequential em algorithm to estimate the finite normal mixtures (FNM) model parameters through information geometric alternating projections. Maximum PL estimation of FNM model parameters produces very efficient estimators when the model specification is a good match to the data generation process, which is the case in channel equalization when the selected channel order is correct.; Partial likelihood allows for inclusion of dependent observations and sequential processing in a likelihood framework, hence it allows development of order selection schemes for real time signal processing using information theoretic criteria. We derive penalized partial likelihood (PPL) as the information theoretic criterion for order selection. To solve the difficult and important problem for real-time applications, on-line order selection, we propose a sequential order selection scheme that increases the order estimate gradually. We further develop a new formulation of PPL that eliminates the storage requirement of the sequential scheme and hence is suitable for on-line implementation. A truly on-line order selection procedure is proposed and is successfully applied to FNM model order selection.