i-DEQ: A stable inertial deep equilibrium model for image restoration
Abstract: Accelerated MRI reconstruction is a classical inverse problem. While physics-based approaches have been widely studied, recent advances in machine learning and data-driven methods have led to substantial improvements in reconstruction quality. However, designing models that are both fast and stable, while providing convergence guarantees, remains an open challenge.
Deep Equilibrium Models (DEQs) have recently emerged as a powerful framework for image restoration. They learn problem-adapted regularizations by formulating the reconstruction as a fixed-point, or equilibrium, problem. Thanks to their implicit formulation, DEQs are also memory-efficient, as they avoid storing the intermediate states of a deep network during inference. However, despite their flexibility and expressive power, DEQs often suffer from high computational costs and training instability.
We propose an inertial Deep Equilibrium Model, or i-DEQ, that learns an explicit nonconvex regularization within the DEQ framework. By incorporating momentum into the fixed-point iterations, i-DEQ achieves accelerated convergence rates while retaining convergence guarantees. Moreover, we observe that i-DEQ is significantly more stable during training and more robust to poor initialization than standard DEQs.
Numerical experiments on a range of linear and nonlinear inverse problems, with a particular focus on accelerated MRI reconstruction, demonstrate that i-DEQ achieves reconstruction quality comparable to state-of-the-art methods, while reducing DEQ inference time by a factor of 2 to 5.