| The ability of a man-made or biological system to understand its environment is limited by the methods used to process sensory information. In particular, the data representation is often a critical component of such systems. Neural systems represent sensory information using distributed populations of neurons that are highly redundant. Understanding the role of redundancy in distributed systems is important both to understanding neural systems and to efficiently solving many modern signal processing problems.; This thesis makes contributions to understanding redundant representations in distributed processing systems in three specific areas. First, we explore the robustness of redundant representations by generalizing existing results regarding noise-reduction to Poisson process modulation. Additionally, we characterize how the noise-reduction ability of redundant representation is weakened when we enforce a distributed processing constraint on the system.; Second, we explore the task of managing redundancy in the context of distributed settings through the specific example of wireless sensor and actuator networks (WSANs). Using a crayfish reflex behavior as a guide, we develop an analytic WSAN model that implements control laws in a completely distributed manner. We also develop an algorithm to optimize the system resource allocation by adjusting the number of bits used to quantize messages on each sensor-actuator communication link. This optimal power scheduling yields several orders of magnitude in power savings over uniform allocation strategies that use a fixed number of bits on each communication link.; Finally, we explore the flexibility of redundant representations for sparse approximation. Neuroscience and signal processing both need a sparse approximation algorithm (i.e., representing a signal with few non-zero coefficients) that is physically implementable in a parallel system and produces smooth coefficient time-series for time-varying signals (e.g., video). We present a class of locally competitive algorithms (LCAs) that minimize a weighted combination of mean-squared error and a coefficient cost function. LCAs produce coefficients with sparsity levels comparable to centralized algorithms while being more realistic for physical implementation. The resultant LCA coefficients for video sequences are more regular (i.e., smoother and more predictable) than the coefficients produced by existing algorithms. |
