Distributed Adaptive Estimation Algorithms Over Networks

Distributed data collection and analysis over networks are ubiquitous. Distributed estimation over networks is to collaboratively estimate some parameters of interest based on noisy measure-ments collected at nodes distributed over a geographic region. On the one hand, in real application, distributed information processing is confined by the limited storage space of the nodes over the network. On the other hand, a streaming information processing method is adopted by the adaptive information processing, which just need to store little real-time data. Thus, the adaptive infor-mation processing can save storage space. In this paper, the distributed adaptive estimation over networks is considered. Both the linear data model and the nonlinear data model are used to model the measurements.Under the linear data model, the data characteristic and the parameters characteristic are fur-ther exploited. In most distributed adaptive estimation algorithms, the output of the system is assumed to be noisy, while the input data is assumed to be accurate. However, in real-world en-vironment, both of the input and output data may be perturbed by noise. Thus, it is unrealistic to assume that all the entries in the input data are accurate and only those in the output data are corrupted. In the cases of noisy input and output data, the total least-squares (TLS) method has the ability of minimizing the perturbations in both input and output data, and thus provides a bet-ter performance than the least-squares (LS)-based method. For the parameter characteristic, many nature and manmade signals present high level of sparsity. Moveover, it is demonstrated that ex-ploiting the sparsity is able to improve the performance. In this paper, the l1- or l0-norm penalty term is used to exploit the sparsity of the signal. Besides, we consider a general case where the unknown parameter vectors (tasks) for different nodes can be different, which is different from the common single-task problem and is known as the multi-task problem. It is assumed that there are some similarities among these tasks. Thus, the performance may be improved by performing the inter-task cooperation. To improve robustness against different degrees of difference among the tasks, an adaptive inter-task cooperation strategy is proposed. For the distributed multi-task esti- mation problem, the joint sparsity is also considered, and the mixed l2,1- or l2,0-norm is adopted to exploit the same structure information.Under the nonlinear data model, based on the extreme learning machine, the distributed non-linear learning problem is transformed into a linear learning problem, and a distributed extreme learning machine algorithm is proposed for the nonlinear learning problem. Note that the algo-rithm’s ability to construct the nonlinear mapping between the input and the output is unchanged. Based on the above discussion, it is known that the design and performance analysis of distributed nonlinear adaptive algorithm is almost the same as that of the distributed linear adaptive algorithm.For the proposed distributed adaptive estimation algorithms, the theoretical analysis on the mean and mean-square performance is also provided. Note that the theoretical analysis of these al-gorithms is different from, and more difficult than that of the existing distributed algorithms. Thus, the theoretical analysis of these algorithms is the main contribution of this paper. In addition, sev-eral numerical simulations are given to verify the effectiveness and advantages of these proposed algorithms.