Fast, explainable and modular generative AI models for computational imaging by unfolding and distillation of Langevin sampling algorithms

Research Theme

Sensor Signal Processing

Aim

To advance generative AI technology for computational imaging by developing novel neural network architectures that combine the explainability, modularity and flexibility of MCMC-based Bayesian imaging methods with the accuracy and scalability of deep learning techniques.

Objectives

  1. Develop new neural network architectures and supervised training strategies tailored for physics-informed generative image reconstruction and uncertainty quantification, with a focus on improving accuracy, scalability to very large problems (e.g., images of size 1024x1024 pixels or larger), explainability (e.g., with layers/modules that map clearly to instrumental models, data fidelity models and regularisation models) and modularity (e.g., that allow modifying or adjusting instrumental models and noise models during inference, without need for retraining).
  2. Leverage the proposed architectures and the industrial partner’s expertise to co-develop novel Bayesian imaging solutions for a flagship application of interest to the partner.
  3. Develop self-supervised training strategies to fine-tune the proposed architectures directly from measurement data, bypassing the need for ground truth data.

Description

Modern computational imaging method rely increasingly on deep generative models to address challenging inverse problems. A notable example is the LATINO-PRO sampler, which combines a large-scale Stable Diffusion XL image prior trained on over five billion image–text pairs with a Bayesian inversion framework tailored to generative AI, achieving state-of-the-art performance on difficult tasks such as x16 image super-resolution. This project aims to develop novel neural network architectures specialised for generative computational imaging. A key novelty is that the proposed architectures will be derived from modern MCMC samplers such as LATINO and from recent work on unfolding, training and distilling of MCMC algorithms into physics-informed generative neural networks. The resulting models are expected to be extremely fast while preserving the structure, interpretability, and uncertainty quantification capabilities of iterative Bayesian inference. This enables flexible inference, including runtime specification of sensing models and their automatic calibration via empirical Bayesian techniques. The goal is to deliver unprecedented image reconstruction accuracy while addressing critical requirements for quantitative imaging in security and defense, notably trustworthiness and explainability. We will consider applications to thermal, multispectral, or low-photon imaging, with possible extensions to dynamic imaging and video reconstruction. For scenarios where reliable ground-truth data are unavailable for training, we will investigate pre-training strategies using public or synthetic datasets, followed by self-supervised fine-tuning directly on sensor data.

Further information

The proposed project builds on the recent paper “Learning few-step posterior samplers by unfolding and distillation of diffusion models” https://arxiv.org/abs/2507.02686, which applies deep unfolding to the LATINO sampler https://arxiv.org/abs/2503.12615, delivering remarkably accurate posterior samples in as little as 8 neural function evaluations.

Closing date: 
Apply now

Principal Supervisor

Assistant Supervisor

Eligibility

A UK first-class honors degree (or its international equivalent) in computational mathematics, computer science, electronic engineering, or a closely related discipline. This project also requires strong programming skills and experience with machine learning frameworks (e.g., PyTorch).

Further information on English language requirements for EU/Overseas applicants.

Funding

Full funding is available for this position.