Theoretical results for deep neural networks

Theoretical results for deep neural networks

Johannes Schmidt-Hieber
Leiden University


SCHEDULE Monday 14th January 2019
21st January 2019
De 14h à 16h30 Salle 2016
Thursday 17th January 2019
24th January 2019
De 14h à 16h30 Salle 2016

Theoretical results for deep neural networks


Large databases and increasing computational power have recently resulted in astonishing performances of deep neural networks for a broad range of extremely complex tasks, including image and text classification, speech recognition and game playing. These deep learning methods are build by imitating the action of the brain and there are few theoretical results as of yet. To formulate a sound mathematical framework explaining these successes is a major challenge for current research.

The course aims to give an overview about existing theory for deep neural networks with a strong emphasis on recent developments. Core topics are approximation theory and complexity bounds for the function spaces generated by deep networks. Beyond that we also discuss modelling aspects, theoretical results on the energy landscape and statistical risk bounds.


– Anthony, M., and Bartlett, P. L. Neural network learning: theoretical foundations. Cambridge University Press, 1999.
– Bach, F. Breaking the curse of dimensionality with convex neural networks. JMLR. 2017.
– Barron, A. Universal approximation bounds for superpositions of a sigmoidal function. IEEE . 1993.
– Barron, A. Approximation and estimation bounds for artificial neural networks. Machine Learning. 1994.
– Choromanska, A., Henaff, M., Mathieu, M., Arous, G. B., LeCun, Y. The loss surface of multilayer networks. Aistats. 2015.
– Goodfellow, Bengio, Courville. Deep Learning. MIT Press, 2016.
– Pinkus, A. Approximation theory of the MLP model in neural networks. Acta Numerica, 143-195, 1999.
– Schmidt-Hieber, J. Nonparametric regression using deep neural networks with ReLU activation function. ArXiv 2017.
– Telgarsky, M. Benefits of depth in neural networks. ArXiv. 2016.
– Yarotsky, D. Error bounds for approximations with deep ReLU networks. Neural Networks. 2017.

Ecole Polytechnique launches a chair on asset management with the group Edmond de Rothschild

The global economy is facing new challenges, among which the growing complexity and rapidly changing financial markets are of utmost importance. In this context, École Polytechnique has teamed up with the Edmond de Rothschild Risk Foundation and Group to create an asset management chair, led by Guillaume Hollard, a researcher at CREST and professor in economics at Polytechnique. This chair was launched on Monday, June 12, 2017. Among the activities of research and teaching supported by the Chair, note in particular the creation of a dedicated investment fund that will allow students to test in real life strategies of investment and to measure the risks. More broadly, the chair aims to remain attentive to changes in practices in the financial industry to better prepare future professionals.

The Malinvaud prize awarded to Pierre Boyer, professor at Ecole Polytechnique

Pierre Boyer, professor at École polytechnique, received on June 21, 2017 the Malinvaud Prize awarded by the French Association of Economics (AFSE) for its article “Efficiency, welfare and political competition”, published in the Quarterly Journal of Economics.

For the second consecutive year, the Malinvaud Prize of the French Association of Economics was awarded to a professor at École polytechnique. This prize is awarded every year since 2010 to a young economist under 40, author of the best scientific article published in a reference journal. In 2016, Isabelle Méjean, Associate Professor at École Polytechnique, was awarded for her article “Firms, Destinations and Aggregate Fluctuations” published in Econometrica. This year, Pierre Boyer is distinguished for his work on the political determinants of income tax reforms.

In his article co-written with Felix Bierbraueur (University of Cologne), Pierre Boyer seeks to understand how the tax rates of the income tax are determined. “There are big differences between the theory developed in public economics, and what we observe in the real world,” says the professor. According to this theory, the tax scale would be chosen by a benevolent planner, that is to say, an abstract entity allocating social weights to different categories of the population. And according to this entity, more social weight would be allocated to low-income populations. “These social weights we are interested in play a key role in setting the balance between fairness and efficiency in the tax scheme of income tax,” says Pierre Boyer. Thus, to explain these differences between theory and reality, the economist proposes to determine the social weight no longer from this theoretical hypothesis, but, this time, from electoral competition by appealing to the political economy .

In their article, and for the first time in this literature, the two economists have combined notions from public economy with political economy. “We managed to make the connection between these two branches thanks to the results of the mathematician Emile Borel, a polytechnician who developed a game of military strategy, says the professor. This game inspired Roger Myerson, winner of the 2007 Nobel Prize in Economics, who introduced him to the study of political economy. From this game, and making an analogy with politics, the young economist and his co-author managed to make the link between these two subfields of economics. Moreover, their article opens the way to practical applications that should ultimately help to better understand the determinants of income tax changes since its creation in 1914 in France, and to explain its changes over years.

“This award is a great encouragement for the pursuit of my research,” says Pierre Boyer who wishes to express his gratitude to the members of the jury and the president, Roger Guesnerie, for having selected his article.

About Pierre Boyer

Pierre Boyer is currently assistant professor at École polytechnique and CREST. He is also a research associate at the CESifo Institute in Munich and at the Research Center on the Political Economy of Reforms at the University of Mannheim in Germany. He holds a doctorate in economics from the Toulouse School of Economics and the École des Hautes Etudes en Sciences Sociales. His research has been rewarded with the thesis prize of the French Association of Economics, the Economics Prize of the Academy of Sciences, Inscriptions and Belles-Lettres of Toulouse, and the Public Economics Prize of the CESifo Institute of Munich (Germany).

Francis Kramarz received an ERC advanced grant

After the starting grant obtained by Isabelle Méjean last year, Francis Kramarz obtained an advanced grant of €1.75M for his project “FIRMNET” on firms and their networks. The project plans to study how social and business networks help companies be resilient in the face of shocks. The research, both theoretical and empirical in France and Sweden, is set to provide insights about labor economics, international trade, econometrics and management.

Isabelle Méjean nominated for the best young French economist award

The Best Young Economist of France Award was created in 2000 by the Cercle des économistes and Le Monde and is awarded each year to a French economist under forty for his/her expertise and contribution to the public debate. On May 22nd, 2017, Antoine Bozio, researcher at the Paris School of Economics and director of IPP, received the prize. Three other economists have been nominated, among which Isabelle Mejean, Associate Professor in Ecole Polytechnique and a member of CREST.