1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.
Processing, storing, and communicating information that originates as an analog signal involves converting this information to bits. This conversion can be described by the combined effect of sampling and quantization, as shown in Figure 1. The digital representation is achieved by first sampling the analog signal to represent it by a set of discretetime samples and then quantizing these samples to a finite number of bits. Traditionally, these two operations are considered separately. The sampler is designed to minimize the information loss due to sampling based on characteristics of the continuoustime input. The quantizer is designed to represent the samples as accurately as possible, subject to a constraint on the number of bits that can be used in the representation. The goal of this article is to revisit this paradigm by illuminating the dependency between these two operations. In particular, we explore the requirements of the sampling system subject to the constraints on the available number of bits for storing, communicating, or processing the analog information.
As a motivation for optimizing sampling and quantization together, consider the minimal sampling rate that arises in classical sampling theory due to Whittaker, Kotelnikov, Shannon, and Landau –. These works establish the Nyquist rate, or the spectral occupancy of the signal, as the critical sampling rate, above which the signal can be perfectly reconstructed from its samples. This statement, however, focuses only on the critical sampling rate required to perfectly reconstruct a bandlimited signal from its discrete samples. It does not incorporate the quantization precision of the samples and does not apply to signals that are not bandlimited. It is, in fact, impossible to obtain an exact representation of any continuous-amplitude sequence of samples by a digital sequence of numbers because of finite quantization precision, and, therefore, any digital representation of an analog signal is prone to error. That is, no continuous-amplitude signal can be reconstructed from its quantized samples with zero distortion regardless of the sampling rate, even when the signal is bandlimited.
The minimal distortion achievable in the presence of quantization depends on the particular way the signal is quantized or, more generally, encoded into a sequence of bits. Since we are interested in the fundamental distortion limit in recovering an analog signal from its digital representation, we consider all possible encoding and reconstruction (decoding) techniques. As an example, in Figure 1, the smartphone display may be viewed as a reconstruction of the real-world painting The Starry Night from its digital representation. No matter how excellent the quality of a smartphone’s high-definition screen may be, this recovery is not perfect, since the digital representation of the analog image is not accurate due to a loss of information occurring during the conversion from analog to bits. Our goal is to analyze this loss as a function of hardware limitations on the sampling mechanism and the number of bits used in the encoding. It is convenient to normalize this number of bits by the signal’s free dimensions, i.e., the dimensions along which new information is generated. For example, the free dimensions of a visual signal are usually the horizontal and vertical axes of the frame, and the free dimension of an audio wave is time. For simplicity, we consider analog signals with a single free dimension, i.e., time. Therefore, our restriction on the digital representation is given in terms of its bit rate - the number of bits per unit time