FUNDAMENTAL LIMITS OF DEEP NEURAL NETWORK LEARNING – Helmut Bölcskei, ETH Zurich



SCHEDULE
Thursday  06th January 2022
Tuesday    11th  January 2022
Thursday  13th  January 2022
Tuesday    18th  January 2022
From 2 PM  to 4:30 PM
From 3 PM  to 5:30 PM
From 2 PM  to 4:30 PM
From 3 PM  to 5:30 PM

SUMMARY:
This short course develops the fundamental limits of deep neural network learning from first principle by characterizing what is possible if no constraints on the learning algorithm and on the amount of training data are imposed. Concretely, we consider Kolmogorov-optimal approximation through deep neural networks with the guiding theme being a relation between the complexity of the function (class) to be approximated and the complexity of the approximating network in terms of connectivity and memory requirements for storing the network topology and the associated quantized weights. The theory we develop educes remarkable universality properties of deep networks. Specifically, deep networks are optimal approximants for markedly different function classes such as affine (i.e., wavelet-like) systems and Weyl-Heisenberg systems. This universality is afforded by a concurrent invariance property of deep networks to time-shifts, scalings, and frequency-shifts. In addition, deep networks provide exponential approximation accuracy—i.e., the approximation error decays exponentially in the number of non-zero weights in the network—of the multiplication operation, polynomials, sinusoidal functions, certain smooth functions, and even one-dimensional oscillatory textures and fractal functions such as the Weierstrass function, the latter two of which do not have previously known methods achieving exponential approximation accuracy. We also show that in the approximation of sufficiently smooth functions finite-width deep networks require strictly smaller connectivity than finite-depth wide networks.

The mathematical concepts forming the basis of this theory, namely metric entropy, linear and nonlinear approximation theory, best M-term approximation, and the theory of frames, will all be developed in the course.

LITERATURE:
https://www.mins.ee.ethz.ch/pubs/p/deep-it-2019
https://www.mins.ee.ethz.ch/pubs/p/deep-approx-18
https://www.mins.ee.ethz.ch/pubs/p/frameschapter

Market, Fighting,Tacit Collusion: The French Mobile Telecommunications Market


Authors:
Marc Bourreau: Professor of Economics at Telecom Paris, academic co-director of the CERRE, and a member of DGComp’s EAGCP
Yutec Sun: Assistant Professor of Economics, CREST-ENSAI
Franck Verboven: Professor of Economics at KU Leuven, Managing Editor of IJIO, Member of IO @ Leuven, Research Fellow at CEPR

Abstract:
We study a major new entry in the French mobile telecommunications market, followed by the introduction of fighting brands by the three incumbents. Using an empirical oligopoly model, we find that the incumbents’ fighting brand strategies are difficult to rationalize as unilateral best responses. Instead, their strategies are consistent with a breakdown of tacit semi-collusion: before entry, the incumbents could successfully coordinate on restricting product variety to avoid cannibalization; after entry, this outcome became harder to sustain because of increased business stealing incentives. Consumers gained considerably from the added variety and, to a lesser extent, from the incumbents’ price responses.

American Economic Review (Forthcoming)

Link:
https://www.aeaweb.org/articles?id=10.1257/aer.20190540&&from=f

ICML 2021: Congratulations to our Researchers


Crest papers accepted at the international Conference on Machine Learning (ICML)

The Crest is pleased to present the work of its researchers and professors at the 38th International Conference on Machine Learning (ICML) being held this week (July 18-24, 2021).

The ICML conference is world-renowned for presenting and publishing cutting-edge research on all aspects of machine learning, and is one of the fastest growing AI conferences in the world.

Congratulations to our Researchers Marco Cuturi, Anna Korba,  Vianney Perchet, Flore Sentenac, Meyer Scetbon (ENSAE Paris, Institut Polytechnique de Paris) an Romaric Gaudel (ENSAI Rennes).

https://www.hi-paris.fr/2021/07/10/hi-paris-at-icml-2021/