Stanley Chan

The tutorial will be based on https://www.nowpublishers.com/article/Details/CGV-112 ArXiv version is available at: https://arxiv.org/abs/2403.18103 The astonishing growth of generative tools in recent years has empowered many exciting applications in text-to-image generation and text-to-video generation. The underlying principle behind these generative tools is the concept of diffusion, a particular sampling mechanism that has overcome some longstanding shortcomings in previous approaches. While there is an ocean of diffusion papers, Python demo, online blogs, etc, I have found it very difficult to understand the underlying mathematical principles. Not only do these online materials lack substance beyond superficial treatments, but often they just recycle another online source with the same technical holes propagating from one to the other. Many students claim they know the diffusion equations, but when asked deeper about the physical meanings, no one can clearly explain what they are. This ICIP tutorial is based on a 90-page tutorial “Tutorial on Diffusion for Imaging and Vision” I wrote in 2024. The purpose is to explain the concepts as clearly as possible, through first-principle arguments, derivations, proofs, toy examples, and figures. There are five topics in this tutorial: Variational AutoEncoder (VAE) a. Encoder and Decoder b. Evidence Lower Bound (ELBO) c. Reparametrization Denoising Diffusion Probabilistic Model (DDPM) a. Transition Distributions b. DDPM’s Evidence Lower Bound c. Reverse Process d. Training and Inference Score Matching Langevin Dynamics (SMLD) a. Sampling b. Stein’s score functions c. Score-matching techniques Stochastic Differential Equations (SDE) a. Forward SDE b. Reverse SDE c. How DDPM and SMLD can be formulated as SDE Physics and Fokker Planck Equations a. Brownian motion b. Markov properties and the Chapman-Kolmogorov Equation c. Master Equation for dynamical processes d. Kramers-Moyal expansion and Fokker-Planck equation The tutorial will be based on https://www.nowpublishers.com/article/Details/CGV-112 ArXiv version is available at: https://arxiv.org/abs/2403.18103 The astonishing growth of generative tools in recent years has empowered many exciting applications in text-to-image generation and text-to-video generation. The underlying principle behind these generative tools is the concept of diffusion, a particular sampling mechanism that has overcome some longstanding shortcomings in previous approaches. While there is an ocean of diffusion papers, Python demo, online blogs, etc, I have found it very difficult to understand the underlying mathematical principles. Not only do these online materials lack substance beyond superficial treatments, but often they just recycle another online source with the same technical holes propagating from one to the other. Many students claim they know the diffusion equations, but when asked deeper about the physical meanings, no one can clearly explain what they are.