Data Driven Algorithms for the Estimation of Low Rank Signals in Structured Subspace

The estimation of low rank signals in noise is a ubiquitous task in signal processing, communications, machine learning, data science, econometrics, etc. A critical step in the estimation of such low rank signals is to identify the signal subspace from the noise subspace via the singular value decomposition (SVD).;In this thesis, we focus on both the design of new SVD based algorithms for problems arising in array processing and the theoretical analysis of their performance. Common array processing tasks include the estimation of direction of arrival (DOA) and clutter suppression for enhanced target detection via beamforming.;We study the estimation performance of a Multiple Signal Classification (MUSIC), a popular algorithm for DOA estimation, for a single source system when the observations are corrupted by noise and randomly missing samples. We show that the MUSIC DOA estimate is consistent even in the presence of randomly missing samples for the single source case. We present a unified analysis of MUSIC based on random matrix theory that is valid in both the sample rich and deficient regimes, and derive an analytical expression the mean squared error (MSE) performance of the estimate. Our analysis uncovers a phase transition phenomenon, in terms of a critical SNR, and critical fraction of entries observed, below which the MUSIC algorithm breaks down.;We consider the problem of clutter suppression in ordinary space time adaptive processing (STAP), and multiple input, multiple output STAP (MIMO STAP). For the case of single clutter, we propose new algorithms for improved estimation of clutter subspace, which exploits the double Kronecker product structure (STAP) or triple Kronecker product structure (MIMO STAP) of the underlying singular vector. These algorithms are based on a rearrangement operator introduced by Van Loan and Pitsianis. We use random matrix theory based analysis to quantify the relative estimation performance of the algorithms, and their breakdown points. We discover via simulations and analysis that algorithms which first compute the SVD of the data, and then exploit the Kronecker product structure provide better estimation performance than algorithms which first exploit the Kronecker product structure and then use the SVD.;We consider the problem of dominant mode rejection (DMR) beamforming. We derive optimal (rank one case) and approximate (higher rank case) shrinkage weights for clutter subspace estimates computed via the SVD, for improved inverse covariance matrix estimation. We highlight the role of singular vector informativeness in this shrinkage operation. We also use this idea to derive an approximate shrinkage weighting for improved low rank tensor estimation. We propose a data driven algorithms that compute these shrinkage weights for both beamforming and tensor estimation.;Throughout, we validate our analyses with numerical simulations.