Skip to content. | Skip to navigation

Personal tools

Sections

UMR 5672

logo de l'ENS de Lyon
logo du CNRS
You are here: Home / Seminars / Machine Learning and Signal Processing / PnP-Flow: Plug-and-Play Image Restoration with Flow Matching

PnP-Flow: Plug-and-Play Image Restoration with Flow Matching

Ségolène Martin (postdoctoral researcher at TU Berlin)
When Oct 15, 2024
from 01:00 to 02:00
Attendees Ségolène Martin
Add event to calendar vCal
iCal

Ségolène Martin   

Title: PnP-Flow: Plug-and-Play Image Restoration with Flow Matching

Abstract:  In this paper, we introduce Plug-and-Play (PnP) Flow Matching, an algorithm for solving imaging inverse problems. PnP methods leverage the strength of pre-trained denoisers, often deep neural networks, by integrating them in optimization schemes. While they achieve state-of-the-art performance on various inverse problems in imaging, PnP approaches face inherent limitations on more generative tasks like inpainting. On the other hand, generative models such as Flow Matching pushed the boundary in image sampling yet lack a clear method for efficient use in image restoration. We propose to combine the PnP framework with Flow Matching (FM) by defining a time-dependent denoiser using a pre-trained FM model. Our algorithm alternates between gradient descent steps on the data-fidelity term, reprojections onto the learned FM path, and denoising. Notably, our method is computationally efficient and memory-friendly, as it avoids backpropagation through ODEs and trace computations. We evaluate its performance on denoising, super-resolution, deblurring, and inpainting tasks, demonstrating superior results compared to existing PnP algorithms and Flow Matching based state-of-the-art methods.
 

Website: https://segolenemartin.github.io/

In Room M7 101, 1st floor, Monod campus, ENSL.