Diffusion

In generative modeling, diffusion refers to a family of methods that learn to reverse a noise-corruption process. In a common formulation, training progressively adds noise to examples and teaches a model a denoising-related prediction. Generation uses the learned behavior to transform noise into a sample through successive steps.

Training and generation run different processes

Imagine training on images with varying levels of added noise. The model learns patterns that help recover structure. At generation time, it can start from random noise and follow a sampling procedure toward an image. Ho and colleagues' denoising diffusion paper presents a foundational formulation.

The output is not simply an original photograph hidden inside the initial noise. The learned model, conditioning, and sampling procedure shape the result. In conditional generation, text or another image can guide that process; image editing can begin from a representation of a supplied image rather than pure noise.

Pixels are not the only working space

Some systems operate on compressed latent representations and use a decoder to produce pixels. Different formulations predict noise, a cleaned representation, or other related quantities, so one description of an implementation should not be treated as universal.

More refinement steps do not guarantee every requested detail. A convincing product illustration can still have the wrong count, label, or geometry. Check the result against the intended content. Diffusion is an important image-generation family, but it is not the only way to generate images or other media.

Reference: Ho et al.: Denoising Diffusion Probabilistic Models.