Self-Supervised Learning, Equivariance, and model-based NN for Inverse Problems
self-supervised learning algorithms} for inverse problemsDeep Learning and Neural Networks
DL-based methods have impressive performance in imagea and signal processing, yet many challenges persist as black-box DL lacks guarantees in convergence, performance, and w.r.t. ground truth -- all elements critical in scientific or medical contexts. To override these limitations and improve the robustness and explainability of DL (XAI), an overarching theme is that equi/invariance to transformations plays a key role in learning and DL, by stabilizing iterative algorithms, for learning from incomplete data, and to measure uncertainty of the reconstructions.
For multivariate multifractal textures synthesis, we studied how GAN-based generative IA can produce them, showing a critical need to control DL architectures to ensure accurate synthesis .
In XAI, we have studied explainability of learning methods, first in a context of privacy and for gene expression data.
Model-based neural network architectures
We proposed archirectures that incorporate knowledge about the acquisition physics, by unrolling variational optimization algorithms. They reduce drastically the number of learning parameters without sacrificing performance. We developed plug-and-play algorithms using pretrained denoisers for inverse problems, ensuring convergence. We stabilized these methods with equivariance constraints and introduced a multiscale framework for faster convergence. Additionally, we designed fast algorithms for uncertainty estimation in network predictions and analyzed inductive biases in imaging architectures.
Self-supervised learning algorithms for inverse problems
These methods rely on noisy and possibly incomplete measurement data alone, bypassing the need for ground-truth references. We made contributions in understanding necessary and sufficient conditions learning without ground-truth data, both in the case of noisyand incomplete data. We proved that learning from incomplete data is possible as long as the image distribution is invariant to a group of transformations, such as rotations or scalings.
The ANR UNLIP (JCJC, J. Tachella) focuses on extending these results to non-linear inverse problems, such as heavily quantized sensing, audio-declipping, and phase retrieval. The results were distributed to a large audience in the self-supervised learning EUSIPCO24 tutorial.
Many learning-based algorithms were included in the open-source deepinverse library deepinv.github.io which lowers the entry bar for new researchers in the field, and obtained an award for open science software.