- This event has passed.
Ludovic STEPHAN (EPFL) – Feature learning in two-layer neural networks with large gradient steps
Statistical Seminar: Every Monday at 2:00 pm.
Time: 2:00 pm – 3:15 pm
Date: 29th March 2024
Place : 3001
Ludovic STEPHAN (EPFL) – Feature learning in two-layer neural networks with large gradient steps
Abstract:
Feature learning is an important mechanism of neural networks, and an integral part of their advantages over simpler (e.g. kernel) learning methods. In this talk, I will present how this phenomenon occurs in two-layer networks trained with large gradient steps, in which both the batch size and the learning rate grow polynomially with the dimension. In particular, we uncover an occurence of the so-called “staircase” property of learning, where important directions are learned sequentially at each new step.
Organizers:
Jaouad MOURTADA (CREST), Anna KORBA (CREST), Karim LOUNICI (CMAP)
Sponsors:
CREST-CMAP