| Blind source separation (BSS) has been found to be important in a wide range of scientific fields, including speech recognition, wireless communication, signal de-noising and anti-interference, image processing, feature extraction, biomedical engineering, sonar signal processing, hyperspectral imaging, earthquake early warning system, signal encryption. As a result, BSS has embraced a rapid development in the past decades and a large number of methods have been developed to solve this challenging problem in the past decades. Nevertheless, several key theoretical issues remain unsolved, which in turn hinders the further progress of BSS. Particularly, how to perform BSS under very weak sparsity assumption? Regarding the famous FOCUSS algorithm for sparse representation, how about its convergence and the associated convergence rate? Under very week assumptions on source signals, how to perform BSS by incorporating additional nature of signals, for example, nonnegativity of signals? We studied these important topics in this dissertation and made the following main contributions:Firstly, we proposed a systematic analysis on the convergence of the FOCUSS algorithm. On the basis of four fundamental hypotheses, we constructed an auxiliary function and then re-derived the FOCUSS algorithm, upon which we established a rigorous theoretical analysis on the convergence of the FOCUSS algorithm. Our extensive experimental results also show that the FOCUSS algorithm almost always find the optimal solution in no more than 50 iterations, even for large-scale problems (say, M=3000 and N=5000).Secondly, we study the convergence rate of the FOCUSS algorithm. Convergence rate is one key fact to evaluate the efficiency of an algorithm. On the basis of convergence analysis, we studied the convergence rate of the FOCUSS algorithm with p?(0,2). Our results showed that, the FOCUSS algorithm enjoys a superlinear convergence rate of order r=2-p when 0<p<1.The convergence is generally only linear when 1?p<2 Nevertheless, a superlinear convergence can be achieved in some cases when 1?p<2. Finally, numerical simulations were provided to demonstrate the real efficiency of FOCUS S under different conditions and settings.Thirdly, we presented a sparse representation method based on a multilayer architecture of FOCUSS. We detailed the procedure to obtain sparse representation by using the FOCUSS algorithm, and then studied how the local convergence would affect the performance. We found that the Lp-norm based cost function may lead to un-identifiability and ill-posedness in sparse representation. To overcome these disadvantages, we proposed a new cost function on the basis of an approximation of Lp-norm. Because the new cost function is non-convex and hence directly applying the FOCUSS algorithm cannot guarantee the global convergence, even if the latent components are very sparse. To improve the possibility of global convergence, we proposed a multilayer FOCUSS algorithm and demonstrated its favorable performance in MRI reconstruction.Moreover, we proposed a new robust algorithm for blind identification, i.e., the mixing matrix estimation. On one hand, the FOCUSS algorithm relies on the accurate estimation of the mixing matrix. On the other hand,PARAllel FACtor analysis (PARAFAC, also known as Canonical Polyadic Decomposition, CPD) is one of the most powerful tools for blind identification, without the need of strict sparsity assumptions on source signals. However, existing CPD algorithms are prone to sticking into local minima, thereby often leading to unreasonable components and deteriorated the separation performance, which has been one major disadvantage of CPD based blind identification methods. By analyzing the Alternating Least-Squares (ALS) and the Hierarchical ALS (HALS) methods for CPD and inheriting their advantages, we proposed an alternating parallel rank-1 decomposition method. The new algorithm is highly parallelizable and particularly, is able to find the global minima with very high probability. Experimental results are provided to justify the high performance of the proposed algorithm.In addition, we proposed the conic hull fitting based nonnegative matrix factorization algorithm. Based on the conic hull fitting, we investigated the near-separable NMF problem, and illustrated that the separability assumption is equivalent to the special case of k-sparse such that k=1 (i.e.,1-sparse). We have proposed several new NMF algorithms based on the half-hyperplances identification and they can be implemented simply via eigenvalue decomposition (EVD). Comparing with Gillis and Vavais's method, the 1-sparse condition can be potentially relaxed to the much weaker (m-1)-sparse condition. Finally, we demonstrated by experiments that the proposed algorithms considerably outperform the state-of-the-art algorithms in terms of precision.Finally, we proposed a local-smoothness constrained NMF with nonlinear convergence rate (NMF-NCR) to solve spectral decomposition (SD) problem. The proposed NMF-NCR utilizes the alternatively iterative update scheme, i.e., updates one factor matrix while fixing the other one. We proved that the gradients of the cost function with respect to each variable matrix are Lipschitz continuous. And a proximal function is constructed for optimizing the cost function. As a result, the proposed method can achieve a nonlinear convergence rate, much faster than the method with linear convergence rate. |
PDF Full Text Request