| Distributed source coding refers to compression in a network with possibly multiple senders and receivers, such that data, or noisy observations of unseen data, from one or more sources, are separately encoded by each sender and the resulting bit streams transmitted to the receivers, which jointly decode all available transmissions with the help of side information locally available. The joint statistics of the source data, the noisy observations and the side information are known, and exploited in the design of the encoders and decoders. This type of compression arises in an increasing number of applications such as sensor networks, satellite networks, and low-complexity video encoding, where past reconstructed frames are used at the decoder as side information to achieve a rate-distortion performance similar to that of conventional encoders with motion compensation.; Long-standing information-theoretic studies have shown that, under certain conditions and for some cases, the compression performance of distributed coding can be made arbitrarily close to that achieved by joint encoding and decoding of the data from all sources, with access to the side information also at the encoder. This includes the case of lossless coding with multiple encoders and a single decoder, but also the case of high-resolution lossy coding with a single encoder, and a single decoder with side information. While these studies establish coding performance bounds, they do not deal with the design of practical, efficient distributed codecs.; In this work, we investigate the design of two of the main building blocks of conventional lossy compression, namely quantizers and transforms, from the point of view of lossy network distributed coding, striving to achieve optimal rate-distortion performance, assuming that all statistical dependencies are known. Additionally, it is assumed that ideal Slepian-Wolf codecs, lossless distributed codecs with nearly optimal performance, are available for the transmission of quantization indices. We present the optimality conditions which such quantizers must satisfy, together with an extension of the Lloyd algorithm for a locally optimal design. In addition, we provide a theoretical characterization of optimal quantizers at high rates and their rate-distortion performance, and we apply it to develop a theoretical analysis of orthonormal transforms for distributed coding. Both the cases of compressing directly observed data, and the case of compressing noisy observations of unseen data, are considered throughout.; Experimental results for Wyner-Ziv quantization of Gaussian sources show consistency between the rate-distortion performance of the quantizers found by our extension of the Lloyd algorithm, and the performance predicted by the high-rate theory presented. Implementations of distributed transform codecs of clean Gaussian data and noisy images experimentally support the rate-distortion improvement due to the introduction of the discrete cosine transform, predicted in the theory developed. |
