[R] An optimal control perspective on diffusion-based generative modeling

Submitted by julbern t3_yresn4 in MachineLearning

Explore the connection between diffusion models and optimal control 🔥
📖 Paper
🎙Come to our oral at the NeurIPS workshop on score-based methods and let’s discuss how one field can benefit from the other.

https://preview.redd.it/6kc1yvz1i4z91.png?width=2852&format=png&auto=webp&s=b6049a8901adc7077d3cfe51508fee70c8133a97

Highlights:
1️⃣ Log-density of the underlying SDE satisfies a HJB equation.
2️⃣ ELBO follows directly from the verification theorem.
3️⃣ Diffusion-based approach to sample from (unnormalized) densities.
...and more to come!

39

Comments

You must log in or register to comment.

SyrupOverflow t1_ivwbn9b wrote

Mhmm, oh yeah, I understand some of these words

7

windoze t1_ivwnlv9 wrote

Do you have any insights into the connection with Poisson Flow

3

julbern OP t1_ivy9l7n wrote

The generative process in this paper is given by an ODE and the diffusivity coefficient in the corresponding Fokker-Planck equation is thus zero. In this case, the verification theorem basically reduces to the instantaneous change of variables formula (Chen et al., 2018).

On the other hand, the solution to the Poisson equation (with homogeneous Dirichlet boundary condition) considered in the paper also has a stochastic representation based on an SDE with a corresponding stopping time (leading to "walk-on-spheres" methods). It would be quite interesting to merge these viewpoints.

1

Benlus t1_ivyk3bx wrote

Is the code already available somewhere? The applications of the unnormalized sampling are very interesting!

2

julbern OP t1_ivynnso wrote

Thank you for your interest! We are in the process of adding more experiments and extending the code and will release it afterwards.

2

Benlus t1_ivyobom wrote

Nice, Ill keep an eye out for the release! Im working in Bayesian Stats so all the theory presented here seems very intriguing. Could you recommend a starting point to read more about control theory?

2

julbern OP t1_ivyy0g1 wrote

The result from stochastic optimal control that we use in the paper ("verification theorem") originates mainly from the work of M. Pavon (1989) and P. Dai Pra (1991).

Perhaps it is best to start with the lecture notes of R. Van Handel (2007).

For books on the topic, I can further suggest:

  1. W. H. Fleming and R. W. Rishel (1975)
  2. W. H. Fleming and H. M. Soner (2006)
  3. H. Pham (2009)

Some more recent works in this direction are the following:

  1. B. Tzen and M. Raginsky (2019)
  2. N. Nüsken and L. Richter (2021)
  3. M. Pavon (2022)
3