BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CREST - ECPv5.1.3//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:CREST
X-ORIGINAL-URL:https://crest.science
X-WR-CALDESC:Events for CREST
BEGIN:VTIMEZONE
TZID:Europe/Helsinki
BEGIN:DAYLIGHT
TZOFFSETFROM:+0200
TZOFFSETTO:+0300
TZNAME:EEST
DTSTART:20240331T010000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0300
TZOFFSETTO:+0200
TZNAME:EET
DTSTART:20241027T010000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=Europe/Helsinki:20240129T140000
DTEND;TZID=Europe/Helsinki:20240129T151500
DTSTAMP:20260710T223428
CREATED:20231222T093753Z
LAST-MODIFIED:20240118T150210Z
UID:16387-1706536800-1706541300@crest.science
SUMMARY:Gabriel PEYRE (Ecole Normale Supérieure) - Conservation Laws for Gradient Flows
DESCRIPTION:Statistical Seminar: Every Monday at 2:00 pm.\nTime: 2:00 pm – 3:15 pm\nDate: 29th January of 2024\nPlace : 3001 \n  \nGabriel PEYRE (Ecole Normale Supérieure) – Conservation Laws for Gradient Flows \n  \nAbstract: \nUnderstanding the geometric properties of gradient descent dynamics is a key ingredient in deciphering the recent success of very large machine learning models. A striking observation is that trained over-parameterized models retain some properties of the optimization initialization. This “implicit bias” is believed to be responsible for some favorable properties of the trained models and could explain their good generalization properties. In this talk I will first rigorously expose the definition and basic properties of “conservation laws”\, which are maximal sets of independent quantities conserved during gradient flows of a given model (e.g. of a ReLU network with a given architecture) with any training data and any loss. Then I will explain how to find the exact number of these quantities by performing finite-dimensional algebraic manipulations on the Lie algebra generated by the Jacobian of the model. In the specific case of linear and ReLu networks\, this procedure recovers the conservation laws known in the literature\, and prove that there are no other laws. \nThe associated paper can be found here https://arxiv.org/abs/2307.00144 and the open source code is here https://github.com/sibyllema/Conservation_laws.This is a joint work with Sibylle Marcotte and Rémi Gribonval. \n  \nOrganizers:\nCristina BUTUCEA (CREST)\, Anna KORBA (CREST)\, Karim LOUNICI (CMAP) \, Jaouad MOURTADA (CREST) \nSponsors:\nCREST-CMAP \n
URL:https://crest.science/event/gabriel-peyre-ecole-normale-superieure-to-be-announced/
CATEGORIES:Statistics
ATTACH;FMTTYPE=:
END:VEVENT
END:VCALENDAR