BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CREST - ECPv5.1.3//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:CREST
X-ORIGINAL-URL:https://crest.science
X-WR-CALDESC:Events for CREST
BEGIN:VTIMEZONE
TZID:Europe/Paris
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:20190331T010000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:20191027T010000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=Europe/Paris:20190325T140000
DTEND;TZID=Europe/Paris:20190325T151500
DTSTAMP:20260713T101906
CREATED:20190110T144721Z
LAST-MODIFIED:20190110T144721Z
UID:12177-1553522400-1553526900@crest.science
SUMMARY:Tengyuan LIANG (University of Chicago-Booth School of Business) - "New Thoughts on Adaptivity\, Generalization and Interpolation Motivated from Neural Networks"
DESCRIPTION:\nThe Statistical Seminar: Every Monday at 2:00 pm.\nTime: 2:00 pm – 3:15 pm\nDate: 25 th of March 2019\nPlace: Room 3001.\nTengyuan LIANG (University of Chicago-Booth School of Business) – “New Thoughts on Adaptivity\, Generalization and Interpolation Motivated from Neural Networks” \nAbstract: Consider the problem: given data pair (x\, y) drawn from a population with f_*(x) = E[y|x]\, specify a neural network and run gradient flow on the weights over time until reaching any stationarity. How does f_t\, the function computed by the neural network at time t\, relate to f_*\, in terms of approximation and representation? What are the provable benefits of the adaptive representation by neural networks compared to the pre-specified fixed basis representation in the classical nonparametric literature? We answer the above questions via a dynamic reproducing kernel Hilbert space (RKHS) approach indexed by the training process of neural networks. We show that when reaching any local stationarity\, gradient flow learns an adaptive RKHS representation\, and performs the global least squares projection onto the adaptive RKHS\, simultaneously. In addition\, we prove that as the RKHS is data-adaptive and task-specific\, the residual for f˚ lies in a subspace that is smaller than the orthogonal complement of the RKHS\, formalizing the representation and approximation benefits of neural networks. \nThen we will move to discuss generalization for interpolating methods in RKHS. In the absence of explicit regularization\, Kernel “Ridgeless” Regression with nonlinear kernels has the potential to fit the training data perfectly. It has been observed empirically\, however\, that such interpolated solutions can still generalize well on test data. We isolate a phenomenon of implicit regularization for minimum-norm interpolated solutions which is due to a combination of high dimensionality of the input data\, curvature of the kernel function\, and favorable geometric properties of the data such as an eigenvalue decay of the empirical covariance and kernel matrices. In addition to deriving a data-dependent upper bound on the out-of-sample error\, we present experimental evidence suggesting that the phenomenon occurs in the MNIST dataset.\nOrganizers:\nCristina BUTUCEA\, Alexandre TSYBAKOV\, Julie JOSSE\, Eric MOULINES\, Mathieu ROSENBAUM\nSponsors:\nCREST-CMAP\n \n\n
URL:https://crest.science/event/jamal-najim-cnrs-upem-tba-2-2-3-5-2-2-2-2-2-2-3-2-2-3/
CATEGORIES:Statistics
ATTACH;FMTTYPE=:
END:VEVENT
END:VCALENDAR