Quantitative CLTs for neural network at random initialization

Categoria: 
Seminari di Probabilità
Data e ora inizio evento: 
Data e ora fine evento: 
Aula: 
Altro (Aula esterna al Dipartimento)
Sede: 

Dipartimento di Matematica, Università di Roma Tor Vergata

Aula esterna: 
Aula Dal Passo
Speaker: 
Lucia Celli (University of Luxembourg)
New quantitative bounds for the convergence of wide feed-forward neural networks to Gaussian processes are presented. For fully connected networks with i.i.d. weights and Gaussian biases, convergence at rate $O(n^{-1/2})$ is established in the Kolmogorov and Wasserstein distances when the network is evaluated at a single input, and in the convex distance when it is evaluated at multiple inputs, as the hidden-layer width $n \to \infty$. The bounds are explicit in the hyperparameters of the network and they give us the possibility to study the infinite depth-width limit. The main tool is Stein's method, following the approach of Chatterjee (2008) and subsequent extensions by Kasprzak, Peccati (2022) and Lachieze-Rey, Peccati (2017). The talk will focus on the proof of the Kolmogorov distance bound, where a key step consists in solving a Stein equation associated with the square of a Lipschitz function.
Back to top