Robust covariance matrix estimation for radar space-time adaptive processing (STAP)

Estimating the disturbance or clutter covariance is a centrally important problem in radar space time adaptive processing (STAP) since estimation of the disturbance or interference covariance matrix plays a central role on radar target detection in the presence of clutter, noise and a jammer. The disturbance covariance matrix should be inferred from training sample observations in practice. Traditional maximum likelihood (ML) estimators are effective when homogeneous (target free) training data is abundant but lead to poor estimates, degraded false alarm rates, and detection loss in the regime of limited training. However, large number of homogeneous training samples are generally not available because of difficulty of guaranteeing target free disturbance observation, practical limitations imposed by the spatio-temporal nonstationarity, and system considerations. The problem has been exacerbated by recent advances that have led to more antenna elements (J) and higher temporal resolution (P) time epochs resulting in a large dimension (N = JP).;In this dissertation, we look to address the aforementioned challenges by exploiting physically inspired constraints into ML estimation. While adding constraints is beneficial to achieve satisfactory performance in the practical regime of limited training, it leads to a challenging problem. Unlike unconstrained estimators, a vast majority of constrained radar STAP estimators are iterative and expensive numerically, which prohibits practical deployment. We focus on breaking this classical trade-off between computational tractability and desirable performance measures, particularly in training starved regimes. In particular, we exploit both the structure of the disturbance covariance and importantly the knowledge of the clutter rank to yield a new rank constrained maximum likelihood (RCML) estimator of clutter/disturbance covariance. We demonstrate that the rank-constrained estimation problem can in fact be cast in the framework of a tractable convex optimization problem, and derive closed form expressions for the estimated covariance matrix. In addition, we derive a new covariance estimator for STAP that jointly considers a Toeplitz structure and a rank constraint on the clutter component. Past work has shown that in the regime of low training, even handling each constraint individually is hard and techniques often resort to slow numerically based solutions. Our proposed solution leverages the rank constrained ML estimator (RCML) of structured covariances to build a computationally friendly approximation that involves a cascade of two closed form solutions. Performance analysis using the KASSPER data set (where ground truth covariance is made available) shows that the proposed RCML estimator vastly outperforms state-of-the art alternatives even for low training including the notoriously difficult regime of K ≤ N training regimes and for the experiments considering real-world scenarios such as target detection performance and the case that some of training samples are corrupted by target information.;Finally, we address the problem of working with inexact physical radar parameters under a practical radar environment. As shown in this dissertation, employing practical constraints such as a rank of the clutter subspace and a condition number of disturbance covariance leads to a practically powerful estimator as well as a closed form solution. While the rank and the condition number are very effective constraints, often practical non-ideality makes it difficult to be known precisely using physical models. We propose a robust covariance estimation method via an expected likelihood (EL) approach. We analyze covariance estimation algorithms under three different cases of imperfect constraints: 1) only rank constraint, 2) both rank and noise power constraint, and 3) condition number constraint. For each case, we formulate estimation of the constraint as an optimization problem with the expected likelihood criterion and formally derive and prove a significant analytical result such as uniqueness of the solution. Through experimental results from a simulation model and the KASSPER data set, we show the estimator with optimal constraints obtained by the EL approach outperforms alternatives in the sense of a normalized signal-to-interference and noise ratio (SINR).