Gaussian bounds for discrete entropies
Abstract
It is well known that the Gaussian distribution has the largest differential entropy amongst all distributions of equal variance. In this paper, we derive similar (generalized) Gaussian upper bounds for discrete (Rényi) entropies of integer-valued variables. Using a mixed discrete-continuous bounding technique and the Poisson summation formula from Fourier analysis, it is proved that in many cases, such Gaussian bounds hold with an additive term that vanishes exponentially as the variance increases.
Origin : Files produced by the author(s)