Simone Rossi
← All publications

Continuous-Time Functional Diffusion Processes

  • Diffusion Models
  • Generative Modeling

We formulate continuous-time diffusion in function spaces, derive a variational training objective, and connect the theory to finite evaluations for practical generation.

Paper NeurIPS 2023

Giulio Franzese1, Giulio Corallo1, Simone Rossi2, Markus Heinonen3, Maurizio Filippone1, Pietro Michiardi1

  1. EURECOM
  2. Stellantis
  3. Aalto University
PublicationPDFPreprintCode

An image can be viewed as a function from spatial coordinates to color values. A sound can be viewed as a function of time. Functional Diffusion Processes (FDPs) formulate noising and generation directly over such functions, then connect those dynamics to the finite evaluations used in computation.

Diffusion over functions

Standard score-based models evolve finite-dimensional vectors. We define a forward stochastic process in a Hilbert space, where each state is a function. This provides a common mathematical setting for continuous data and functional perturbations, including noising and blurring.

Generating a function requires reversing these dynamics. We derive the reverse process using an infinite-dimensional counterpart of the score. The reference probability measure and the covariance operator of the functional noise determine how this score enters the reverse dynamics.

A variational training objective

A practical model needs both reverse dynamics and a learnable objective. We use an infinite-dimensional version of Girsanov's theorem to derive an evidence lower bound (ELBO). This connects score approximation to a variational objective for the functional generative model.

The ELBO makes score learning an optimization problem over functions. To train a network, we evaluate this objective through discrete observations of those functions.

From functions to finite evaluations

We extend a sampling-theorem argument to identify classes of functions that can be recovered from countably many evaluations. This states the conditions under which discrete observations retain the underlying functional information.

The resulting implementation uses finite-dimensional approximations to the forward process, reverse process, and ELBO. We explore implicit neural representations and transformer-based mappings for representing the functions and their scores. Coordinates and function values provide the interface between the continuous object and the network.

Generating images with a simple network

Face images generated by a Functional Diffusion Process.
Face samples generated by a Functional Diffusion Process. The experiment illustrates how a model defined over continuous functions can produce image data through finite evaluations.

In our image experiments, an FDP generates realistic samples using a multilayer perceptron with substantially fewer parameters than the conventional diffusion baselines in the comparison. This architecture represents both the function and its score through coordinate evaluations.

Generation combines a function-space model with a finite set of evaluation points. The reconstruction assumptions specify which functional information those evaluations preserve, while the finite-dimensional approximation determines the training and sampling procedure.