BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CREST - ECPv5.1.3//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:CREST
X-ORIGINAL-URL:https://crest.science
X-WR-CALDESC:Events for CREST
BEGIN:VTIMEZONE
TZID:Europe/Helsinki
BEGIN:DAYLIGHT
TZOFFSETFROM:+0200
TZOFFSETTO:+0300
TZNAME:EEST
DTSTART:20210328T010000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0300
TZOFFSETTO:+0200
TZNAME:EET
DTSTART:20211031T010000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=Europe/Helsinki:20211103T140000
DTEND;TZID=Europe/Helsinki:20211103T150000
DTSTAMP:20260725T212226
CREATED:20211025T125542Z
LAST-MODIFIED:20211027T080027Z
UID:13089-1635948000-1635951600@crest.science
SUMMARY:Tam Lê (TSE) – “Nonsmooth implicit differentiation for Machine Learning and Optimization”
DESCRIPTION:\nStatistics-Econometrics-Machine Learning Seminar. \nTime: 2:00 pm – 3:00 pm\nDate: 3rd of november 2021\nPlace: room 3001 and Online \nTam Lê (TSE) – “Nonsmooth implicit differentiation for Machine Learning and Optimization” \nAbstract : The Implicit Function Theorem (IFT) has found several applications in machine learning such as hyperparameter optimization or implicit neural networks training. We will review the IFT and its extensions to nonsmooth functions. The IFT first ensures the uniqueness and regularity of the implicit function and then provides a calculus i.e. a formula to compute the gradient. Its extensions to the nonsmooth case still provide uniqueness and regularity but no calculus. Recalling usual tools from nonsmooth analysis such as the Clarke Jacobian\, we will explain with an example why they fail to generalize the implicit differentiation formula. We propose a solution to this issue using the recent notion of conservative Jacobian. This is a notion of Jacobian for nonsmooth functions compatible with the compositional rules of differential calculus which justifies why we can replace the derivatives by Clarke Jacobians in the implicit differentiation formula. We will see how this theory allows to obtain a convergence result for gradient descents implemented with implicit differentiation. We conclude the talk showcasing pathological training trajectories one can have when implicit differentiation is applied naively. \nSponsors: \n\n\n\nCREST \n\n\n
URL:https://crest.science/event/tam-le-tse-nonsmooth-implicit-differentiation-for-machine-learning-and-optimization/
LOCATION:zoom
END:VEVENT
END:VCALENDAR