Spectral norm estimation in deep learning
| Quand ? |
Le 06/06/2024, de 13:00 à 14:00 |
|---|---|
| Participants |
Blaise Delattre |
| Ajouter un événement au calendrier |
vCal
iCal
|
Blaise Delattre
Title: Spectral norm estimation in deep learning
Abstract: This work leverages the use of Gram iteration, an efficient, deterministic, and differentiable method for computing spectral norm with an upper bound guarantee. Designed for dense matrix and circular convolutional layers, we generalize the use of the Gram iteration to zero padding convolutional layers and prove its quadratic convergence. We also provide theorems for bridging the gap between circular and zero padding convolution's spectral norm. We design a spectral rescaling that can be used as a competitive -Lipschitz layer that enhances network robustness. Demonstrated through experiments, our method outperforms state-of-the-art techniques in precision, computational cost, and scalability.
Website: https://scholar.google.com/citations?hl=fr&user=0SNA45sAAAAJ&view_op=list_works&sortby=pubdate
In Room M7.101 of Monod campus, ENSL.