The Statistical Seminar: Every Monday at 2:00 pm.
Time: 2:00 pm – 3:15 pm
Date: 16th of September 2019
Place: Room 3001.
Yuri POLYANSKI (MIT) – “Smoothed Empirical Measures and Entropy Estimation“

Abstract: In this talk we discuss behavior of the empirical measure P_n corresponding to iid samples from a distribution P on a d-dimensional space. Let Q_n and Q denote the result of convolving P_n and P, respectively, with an isotropic standard Guassian kernel. We discuss convergence of the p-Wasserstein, KL and other distances between Q_n and Q. Curiously, for some distances (like 1-Wasserstein) we get parametric 1/sqrt(n) speed of convergence regardless of dimension, whereas for some other distances (like 2-Wasserstein) the 1/sqrt(n) rate can change to \omega(1/sqrt(n)). We give an if and only if characterization in the class of subgaussian P for the parametric rate. As an application, we show that differential entropy of Q_n converges to that of Q at parametric rate 1/sqrt(n) regardless of dimension. An estimator of differential entropy of Q, in turn, allows us to estimate the input-output mutual information in noisy neural networks.

Joint work with Ziv Goldfeld, Kristjan Greenewald and Jonathan Weed.
Organizers:
Cristina BUTUCEA, Alexandre TSYBAKOV, Julie JOSSE, Eric MOULINES, Mathieu ROSENBAUM
Sponsors:
CREST-CMAP