Beyond ICA: Advances on temporal BYY learning, state space modeling and blind signal processing

Blind source separation (BSS) is to recover a set of statistically independent source signals (shortly as sources) from observed mixtures of them while the mixing process is unknown. Due to its attractive applications in speech processing, wireless communications, time series analysis and so on, this problem with a linear instantaneous mixture has been formulated as independent component analysis (ICA), widely studied in the past decade. However, the ICA requires that: (1) each source is independently and identically distributed (i.i.d.) signal; (2) sources are generally non-Gaussian distributed with at most one Gaussian signal; (3) observations are measured without the presence of measurement noise. These pre-conditions all limit the BSS potential applications.; Recently, (Xu 2000) has developed a temporal Bayesian Ying-Yang (TBYY) learning system that models temporal sources and observations in general state-space equations. In the TBYY system, a temporal factor analysis (TFA) has been proposed for identifying real generally-distributed sources in the presence of measurement noise, which therefore includes ICA as a special case. Furthermore, the TBYY system also extends hidden Markov model (HMM) to independent HMM for discrete source separation. In this thesis, we aim at exploring the BSS approaches in a linear state-space model within the framework of TFA and independent HMM, respectively. In the former, we further investigate the TFA problem through the TBYY system. Firstly, we implement an integral-approximated algorithm (Xu 2000) (shortly as TFA-A) for the Gaussian case of TFA, and compare it with non-temporal one to show the importance of considering temporal relationship in source identification. Then, we set up a connection between the TFA and the traditional filtering problem in control theory, and therefore obtain a new TFA algorithm (shortly as TFA-P) without using any approximation technique. This precise algorithm adaptively estimates the sources and their variances by Kalman filter instead of the gradient-based method used in the approximate algorithm (Xu 2000), thus resulting in better performance in general as demonstrated by the experiments. Although both of the TFA-A and TFA-P algorithms are derived from the Gaussian-case TFA, they are actually a first-order and a quadratic approximate algorithm respectively for the non-Gaussian case of TFA. We have therefore applied them to the TFA on general sources for further comparisons with satisfactory results obtained. In the latter, we set up the connection between the state identification and the observation clustering features, whereby an approach based on Rival Penalized Competitive Learning (Xu et al. 1992, 1993) rule is proposed to perform discrete source recovery without any knowledge about the number of sources.; Moreover, we also provide a new aspect on the TFA model without measurement noise by presenting a dual multivariate auto-regressive (AR) modeling for temporal source separation. In this modeling, both of sources and observations are modeled as a multivariate AR process, respectively. Consequently, the mixing process from temporal sources to observations is the same as that from the independent residuals of the source AR (SAR) process to those of the observation AR (OAR) process. We can therefore avoid the source temporal effects in performing BSS by learning the de-mixing system on the OAR residuals rather than the observations. Particularly, we implement this approach by modeling each source signal as a finite mixture of generalized autoregressive conditional heteroskedastic (LARCH) process (Bollerslev 1986). An adaptive algorithm is proposed to appropriately extract the OAR residuals based on maximum-likelihood learning, together with learning the de-mixing system by using an existing non-temporal ICA algorithm. The...