Statistical Seminar: 
Time: 2:15 pm – 3:15 pm
Date: 13th of April 2023
Place: Room 3001 + ZOOM

Matus TELGARSKY (Université Illinois, Urbana Champaign) – “Searching for the implicit bias of deep learning”

Abstract:

What makes deep learning special — why is it effective in so many settings where other models fail? This talk will present recent progress from three perspectives. The first result is approximation-theoretic: deep networks can easily represent phenomena that require exponentially-sized shallow networks, decision trees, and other classical models. Secondly, I will show that their statistical generalization ability — namely, their ability to perform well on unseen testing data — is correlated with their prediction margins, a classical notion of confidence. Finally, comprising the majority of the talk, I will discuss the interaction of the preceding two perspectives with optimization: specifically, how standard descent methods are implicitly biased towards models with good generalization. Here I will present two approaches: the strong implicit bias, which studies convergence to specific well-structured objects, and the weak implicit bias, which merely ensures certain good properties eventually hold, but has a more flexible proof technique.

Link :  https://zoom.us/j/91051481144?pwd=alF1cjJUZ0pmUlprRmJjUWRDNU9odz09  

ID de réunion : 910 5148 1144
Code secret : 590133

Organizers:
Cristina BUTUCEA (CREST), Alexandre TSYBAKOV (CREST), Karim LOUNICI (CMAP) , Jaouad MOURTADA (CREST)
Sponsors:
CREST-CMAP