Claude Shannon

Tuesday, November 17th, 2015

Room 202 in Packard Bldg., Stanford University
Parking Generally Free In Nearby Lots After 4:00 pm

Refreshments and Conversation at 6:00 P.M.
Presentation at 6:30 P.M.

Reservation Suggested

Rate-distortion of sub-Nyquist sampled processes

Alon Kipnis
PhD Candidate, Stanford EE Department


Consider the task of analog to digital conversion in which a continuous time random process is mapped into a stream of bits. The optimal trade-off between the bitrate and the minimal average distortion in recovering the waveform from its bit representation is described by the Shannon rate-distortion function of the continuous-time source. Traditionally, in solving for the optimal mapping and the rate-distortion function we assume that the analog waveform has a discrete time version, as in the case of a band-limited signal sampled above its Nyquist frequency. Such assumption, however, may not hold in many scenarios due to wideband signaling and A/D technology limitations. A more relevant assumption in such scenarios is that only a sub-Nyquist sampled version of the source can be observed, and that the error in analog to digital conversion is due to both sub-sampling and finite bit representation. This assumption gives rise to a combined sampling and source coding problem, in which the quantities of merit are the sampling frequency, the bitrate and the average distortion.

In this talk we will characterize the optimal trade-off among these three parameters. The resulting rate-distortion-sampling frequency function can be seen as a generalization of the classical Shannon-Kotelnikov-Whittaker sampling theorem to the case where finite bit rate representation is required. This characterization also provides us with a new critical sampling rate: the minimal sampling rate required to achieve the rate-distortion function of a Gaussian stationary process for a given rate-distortion pair. That is, although the Nyquist rate is the minimal sampling frequency that allows perfect reconstruction of a band-limited signal from its samples, relaxing perfect reconstruction to a prescribed distortion allows sampling below the Nyquist rate while achieving the same rate-distortion trade-off.


Photo of Alon Kipnis Alon Kipnis Kipnis is a fourth year PhD candidate in the Electrical Engineering department at Stanford University. Alon received the B.Sc. degree in mathematics (summa cum laude) and the B.Sc. degree in electrical engineering (summa cum laude) in 2010, and the M.Sc in mathematics in 2012, all from Ben-Gurion University of the Negev. He is currently a Ph.D candidate in the department of electrical engineering at Stanford University. His research interests include information theory, data compression and stochastic analysis.



SCV IT Society Webmaster (
Last updated on

Return to SCV IT Society Homepage