Diffusion Models: Mathematical Foundations, Score Matching, and PyTorch Architecture

Generative Paradigms and Non-Equilibrium Diffusion

Generative modeling addresses the fundamental task of modeling a high-dimensional probability distribution $p_{\text{data}}(\mathbf{x})$ given empirical samples. Prior frameworks exhibit distinct trade-offs:

1. Generative Adversarial Networks (GANs): Train a minimax game ${\min }_G{\max }_{D}V(D,G)$. While achieving fast single-step inference, adversarial optimization suffers from mode collapse, gradient vanishing, and non-convergent limit cycles.
2. Variational Autoencoders (VAEs): Maximize the Evidence Lower Bound (ELBO) using a single-step encoder $q_{\varphi }(\mathbf{z}\mid \mathbf{x})$ and decoder $p_{\theta }(\mathbf{x}\mid \mathbf{z})$. Single-step decoders average multimodal distributions, causing visual blurriness.
3. Normalizing Flows: Learn bijective mappings $\mathbf{z}=f_{\theta }(\mathbf{x})$ via exact change-of-variables $\log p(\mathbf{x})=\log p(\mathbf{z})+\log |\det {\mathbf{J}}_{f_{\theta }}(\mathbf{x})|$. Enforcing tractable Jacobian determinants severely restricts model expressivity.

Diffusion Probabilistic Models (Sohl-Dickstein et al., ICML 2015; Ho et al., NeurIPS 2020) resolve these limitations by casting synthesis as the time-reversal of a continuous Gaussian noise perturbation process. By decomposing the generative task across $T\approx 1000$ discrete timesteps (or continuous time $t\in [0,1]$), the model learns a sequence of tractable, locally Gaussian denoising transitions.

Architectural Comparison

Mathematical Foundations

Figure 1: Forward perturbation Markov chain $q({\mathbf{x}}_t\mid {\mathbf{x}}_{t-1})$ and learned reverse generative denoising chain $p_{\theta }({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t)$.

1. Forward Gaussian Perturbation Chain

Given data ${\mathbf{x}}_0\sim q({\mathbf{x}}_0)$, the forward diffusion process is defined as a fixed Gaussian Markov chain parameterized by a variance schedule ${\beta }_1,{\beta }_2,\dots ,{\beta }_T\in (0,1)$:

$q({\mathbf{x}}_{1:T}\mid {\mathbf{x}}_0)={\prod }_{t=1}^{T}q({\mathbf{x}}_t\mid {\mathbf{x}}_{t-1}),q({\mathbf{x}}_t\mid {\mathbf{x}}_{t-1})=\mathcal{N}\left({\mathbf{x}}_t;\sqrt{1-{\beta }_t}{\mathbf{x}}_{t-1},{\beta }_t\mathbf{I}\right)$

Let ${\alpha }_t=1-{\beta }_t$ and ${\bar{\alpha }}_t={\prod }_{s=1}^t{\alpha }_s$. Evaluating recursive Gaussian convolutions yields the Closed-Form Marginal Distribution at any timestep $t$:

$q({\mathbf{x}}_t\mid {\mathbf{x}}_0)=\mathcal{N}\left({\mathbf{x}}_t;\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0,(1-{\bar{\alpha }}_t)\mathbf{I}\right)$

${\mathbf{x}}_t=\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0+\sqrt{1-{\bar{\alpha }}_t}\mathbf{ϵ},\mathbf{ϵ}\sim \mathcal{N}(0,\mathbf{I})$

As $t\to T$, ${\bar{\alpha }}_T\approx 0$, and the distribution asymptotically converges to standard isotropic Gaussian noise ${\mathbf{x}}_T\sim \mathcal{N}(0,\mathbf{I})$.

2. Reverse Process and Posterior Derivation

The reverse trajectory $p_{\theta }({\mathbf{x}}_{0:T})=p({\mathbf{x}}_T){\prod }_{t=1}^T{p}_{\theta }({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t)$ parameters Gaussian transitions:

$p_{\theta }({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t)=\mathcal{N}\left({\mathbf{x}}_{t-1};{\mathbf{\mu }}_{\theta }({\mathbf{x}}_t,t),{\mathbf{\Sigma }}_{\theta }({\mathbf{x}}_t,t)\right),p({\mathbf{x}}_T)=\mathcal{N}(0,\mathbf{I})$

When conditioned on the clean data ${\mathbf{x}}_0$, the true reverse step $q({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t,{\mathbf{x}}_0)$ is analytically solvable via Bayes’ expansion:

$q({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t,{\mathbf{x}}_0)=\frac{q({\mathbf{x}}_t\mid {\mathbf{x}}_{t-1},{\mathbf{x}}_0)q({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_0)}{q({\mathbf{x}}_t\mid {\mathbf{x}}_0)}=\mathcal{N}\left({\mathbf{x}}_{t-1};{\tilde{\mathbf{\mu }}}_t({\mathbf{x}}_t,{\mathbf{x}}_0),{\tilde{\beta }}_t\mathbf{I}\right)$

where:

${\tilde{\beta }}_t=\frac{1-{\bar{\alpha }}_{t-1}}{1-{\bar{\alpha }}_t}{\beta }_t$

${\tilde{\mathbf{\mu }}}_t({\mathbf{x}}_t,{\mathbf{x}}_0)=\frac{\sqrt{{\bar{\alpha }}_{t-1}}{\beta }_t}{1-{\bar{\alpha }}_t}{\mathbf{x}}_0+\frac{\sqrt{{\alpha }_t}(1-{\bar{\alpha }}_{t-1})}{1-{\bar{\alpha }}_t}{\mathbf{x}}_t$

3. Variational Lower Bound (ELBO) and ${\mathcal{L}}_{\text{simple}}$ Objective

Optimizing negative log-likelihood $-\log p_{\theta }({\mathbf{x}}_0)$ yields the variational bound:

${\mathcal{L}}_{\text{VLB}}={\mathbb{E}}_q\left[\underset{{\mathcal{L}}_T}{\underbrace{D_{\text{KL}}\left(q({\mathbf{x}}_T\mid {\mathbf{x}}_0)\Vert p({\mathbf{x}}_T)\right)}}+{\sum }_{t=2}^T\underset{{\mathcal{L}}_{t-1}}{\underbrace{D_{\text{KL}}\left(q({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t,{\mathbf{x}}_0)\Vert p_{\theta }({\mathbf{x}}_{t-1}\mid {\mathbf{x}}_t)\right)}}\underset{{\mathcal{L}}_0}{\underbrace{-\log p_{\theta }({\mathbf{x}}_0\mid {\mathbf{x}}_1)}}\right]$

Expressing clean data ${\mathbf{x}}_0=\frac{{\mathbf{x}}_t-\sqrt{1-{\bar{\alpha }}_t}\mathbf{ϵ}}{\sqrt{{\bar{\alpha }}_t}}$ allows parameterizing the neural network mean ${\mathbf{\mu }}_{\theta }({\mathbf{x}}_t,t)$ via a noise predictor ${\mathbf{ϵ}}_{\theta }({\mathbf{x}}_t,t)$:

${\mathbf{\mu }}_{\theta }({\mathbf{x}}_t,t)=\frac{1}{\sqrt{{\alpha }_t}}\left({\mathbf{x}}_t-\frac{{\beta }_t}{\sqrt{1-{\bar{\alpha }}_t}}{\mathbf{ϵ}}_{\theta }({\mathbf{x}}_t,t)\right)$

Substituting into the KL divergence term produces the Simplified Noise-Prediction Loss:

$\boxed{{\mathcal{L}}_{\text{simple}}(\theta )={\mathbb{E}}_{t,{\mathbf{x}}_0,\mathbf{ϵ}}\left[{\Vert \mathbf{ϵ}-{\mathbf{ϵ}}_{\theta }\left(\sqrt{{\bar{\alpha }}_t}{\mathbf{x}}_0+\sqrt{1-{\bar{\alpha }}_t}\mathbf{ϵ},t\right)\Vert }_2^2\right]}$

where $t\sim \mathrm{Uniform}(\{1,\dots ,T\})$. Discarding the analytical weighting factor $\frac{{\beta }_t^2}{2{\sigma }_t^2{\alpha }_t(1-{\bar{\alpha }}_t)}$ places higher emphasis on intermediate perceptual scales, substantially improving sample fidelity.

4. Continuous-Time SDEs and Score Matching Duality

In continuous time ($T\to \infty$), the forward perturbation converges to an Itô Stochastic Differential Equation (SDE):

$\mathrm{d}\mathbf{x}=\mathbf{f}(\mathbf{x},t)\mathrm{d}t+g(t)\mathrm{d}\mathbf{w}$

where $\mathbf{w}$ is standard Brownian motion. For Variance Preserving (VP) diffusion (DDPM equivalent):

$\mathbf{f}(\mathbf{x},t)=-\frac{1}{2}\beta (t)\mathbf{x},g(t)=\sqrt{\beta (t)}$

By Anderson’s Theorem (1982) and Song et al. (ICLR 2021), the Reverse-Time SDE is:

$\mathrm{d}\mathbf{x}=\left[\mathbf{f}(\mathbf{x},t)-g^2(t){\nabla }_{\mathbf{x}}\log p_t(\mathbf{x})\right]\mathrm{d}t+g(t)\mathrm{d}\bar{\mathbf{w}}$

where $\mathrm{d}\bar{\mathbf{w}}$ is reverse Brownian motion and ${\nabla }_{\mathbf{x}}\log p_t(\mathbf{x})$ is the Stein Score Function. By Tweedie’s Identity, the noise prediction directly parameterizes the score:

${\nabla }_{{\mathbf{x}}_t}\log p_t({\mathbf{x}}_t)=-\frac{{\mathbf{ϵ}}_{\theta }({\mathbf{x}}_t,t)}{\sqrt{1-{\bar{\alpha }}_t}}$

Associated with every SDE is a deterministic Probability-Flow ODE sharing identical marginal probability densities $p_t(\mathbf{x})$:

$\mathrm{d}\mathbf{x}=\left[\mathbf{f}(\mathbf{x},t)-\frac{1}{2}{g}^2(t){\nabla }_{\mathbf{x}}\log p_t(\mathbf{x})\right]\mathrm{d}t$

Discretizing this deterministic ODE yields modern accelerated samplers: DDIM, DPM-Solver++, and EDM Heun solvers.

5. Classifier-Free Guidance (CFG)

To direct conditional generation $p(\mathbf{x}\mid \mathbf{c})$ without a separate classifier, Classifier-Free Guidance (Ho & Salimans, 2022) randomly replaces conditioning tokens $\mathbf{c}$ with an empty token $\varnothing$ during training ($p_{\text{uncond}}\approx 0.1\text{–}0.2$). At inference, the guided noise prediction extrapolates away from the unconditional estimate:

${\widehat{\mathbf{ϵ}}}_{\theta }({\mathbf{x}}_t,t,\mathbf{c})={\mathbf{ϵ}}_{\theta }({\mathbf{x}}_t,t,\varnothing )+s\cdot \left({\mathbf{ϵ}}_{\theta }({\mathbf{x}}_t,t,\mathbf{c})-{\mathbf{ϵ}}_{\theta }({\mathbf{x}}_t,t,\varnothing )\right)$

where $s\ge 1.0$ is the guidance scale, trading off sample diversity for sharp alignment with conditioning constraints.

Production PyTorch Implementation

Below is a complete, modular PyTorch implementation of the Gaussian Diffusion engine, U-Net with FiLM Time-Modulation & Spatial Self-Attention, and accelerated DDIM deterministic sampling.

Step 1: Forward/Reverse Gaussian Diffusion Engine

from __future__ import annotations

import math
import torch
import torch.nn as nn
import torch.nn.functional as F


class GaussianDiffusionEngine(nn.Module):
    """Core mathematical engine for Gaussian forward perturbation and reverse sampling."""

    def __init__(self, timesteps: int = 1000, beta_schedule: str = "cosine") -> None:
        super().__init__()
        self.timesteps = timesteps

        if beta_schedule == "linear":
            betas = torch.linspace(1e-4, 0.02, timesteps, dtype=torch.float32)
        elif beta_schedule == "cosine":
            steps = timesteps + 1
            s = 0.008
            x = torch.linspace(0, timesteps, steps, dtype=torch.float32)
            alphas_cumprod = torch.cos(((x / timesteps) + s) / (1.0 + s) * math.pi * 0.5) ** 2
            alphas_cumprod = alphas_cumprod / alphas_cumprod[0]
            betas = torch.clip(1.0 - alphas_cumprod[1:] / alphas_cumprod[:-1], 1e-4, 0.999)
        else:
            raise NotImplementedError(f"Schedule {beta_schedule} not implemented.")

        alphas = 1.0 - betas
        alphas_cumprod = torch.cumprod(alphas, dim=0)
        alphas_cumprod_prev = F.pad(alphas_cumprod[:-1], (1, 0), value=1.0)

        # Register buffers for device auto-transfer
        self.register_buffer("betas", betas)
        self.register_buffer("alphas", alphas)
        self.register_buffer("alphas_cumprod", alphas_cumprod)
        self.register_buffer("sqrt_alphas_cumprod", torch.sqrt(alphas_cumprod))
        self.register_buffer("sqrt_one_minus_alphas_cumprod", torch.sqrt(1.0 - alphas_cumprod))
        
        # Analytical posterior variance \tilde{\beta}_t
        posterior_variance = betas * (1.0 - alphas_cumprod_prev) / (1.0 - alphas_cumprod)
        self.register_buffer("posterior_variance", posterior_variance.clamp(min=1e-20))

    def q_sample(self, x_0: torch.Tensor, t: torch.Tensor, noise: torch.Tensor | None = None) -> tuple[torch.Tensor, torch.Tensor]:
        """Diffuse data x_0 to timestep t: x_t = sqrt(\bar{\alpha}_t)*x_0 + sqrt(1 - \bar{\alpha}_t)*\epsilon."""
        if noise is None:
            noise = torch.randn_like(x_0)
        
        sqrt_alpha_bar = self.sqrt_alphas_cumprod[t].view(-1, 1, 1, 1)
        sqrt_one_minus_alpha_bar = self.sqrt_one_minus_alphas_cumprod[t].view(-1, 1, 1, 1)
        x_t = sqrt_alpha_bar * x_0 + sqrt_one_minus_alpha_bar * noise
        return x_t, noise

    @torch.no_grad()
    def ddim_sample(
        self,
        model: nn.Module,
        shape: tuple[int, ...],
        num_inference_steps: int = 50,
        eta: float = 0.0,
        device: str = "cuda",
    ) -> torch.Tensor:
        """Deterministic ODE / DDIM reverse sampling over subsampled timesteps."""
        step_ratio = self.timesteps // num_inference_steps
        timesteps = (torch.arange(0, num_inference_steps) * step_ratio).round().flip(0).long().to(device)
        
        x = torch.randn(shape, device=device)

        for i, t in enumerate(timesteps):
            t_prev = timesteps[i + 1] if i + 1 < len(timesteps) else -1
            t_tensor = torch.full((shape[0],), t, device=device, dtype=torch.long)
            
            eps_pred = model(x, t_tensor)
            
            alpha_bar_t = self.alphas_cumprod[t]
            alpha_bar_t_prev = self.alphas_cumprod[t_prev] if t_prev >= 0 else torch.tensor(1.0, device=device)

            # Predict original sample x_0
            x_0_pred = (x - torch.sqrt(1.0 - alpha_bar_t) * eps_pred) / torch.sqrt(alpha_bar_t)

            # Variance calculation
            sigma_t = eta * torch.sqrt((1.0 - alpha_bar_t_prev) / (1.0 - alpha_bar_t) * (1.0 - alpha_bar_t / alpha_bar_t_prev))
            dir_xt = torch.sqrt((1.0 - alpha_bar_t_prev - sigma_t**2).clamp(min=0.0)) * eps_pred

            x = torch.sqrt(alpha_bar_t_prev) * x_0_pred + dir_xt
            if eta > 0.0:
                x = x + sigma_t * torch.randn_like(x)

        return x

Step 2: U-Net Backbone with FiLM Conditioning & Self-Attention

from __future__ import annotations

import math
import torch
import torch.nn as nn
import torch.nn.functional as F


class SinusoidalPositionalEmbedding(nn.Module):
    def __init__(self, dim: int) -> None:
        super().__init__()
        self.dim = dim

    def forward(self, t: torch.Tensor) -> torch.Tensor:
        half_dim = self.dim // 2
        embeddings = math.log(10000) / (half_dim - 1)
        embeddings = torch.exp(torch.arange(half_dim, device=t.device) * -embeddings)
        embeddings = t[:, None].float() * embeddings[None, :]
        return torch.cat((embeddings.sin(), embeddings.cos()), dim=-1)


class ResBlockFiLM(nn.Module):
    """Residual Block with Feature-wise Linear Modulation (FiLM)."""

    def __init__(self, in_ch: int, out_ch: int, time_dim: int, groups: int = 8) -> None:
        super().__init__()
        self.norm1 = nn.GroupNorm(groups, in_ch)
        self.conv1 = nn.Conv2d(in_ch, out_ch, kernel_size=3, padding=1)
        self.norm2 = nn.GroupNorm(groups, out_ch)
        self.conv2 = nn.Conv2d(out_ch, out_ch, kernel_size=3, padding=1)
        
        self.time_proj = nn.Linear(time_dim, out_ch * 2) # Outputs Scale & Shift
        self.shortcut = nn.Conv2d(in_ch, out_ch, kernel_size=1) if in_ch != out_ch else nn.Identity()

    def forward(self, x: torch.Tensor, t_emb: torch.Tensor) -> torch.Tensor:
        h = self.conv1(F.silu(self.norm1(x)))
        
        # FiLM Scale & Shift
        scale, shift = self.time_proj(F.silu(t_emb)).chunk(2, dim=-1)
        h = h * (1.0 + scale[:, :, None, None]) + shift[:, :, None, None]
        
        h = self.conv2(F.silu(self.norm2(h)))
        return h + self.shortcut(x)


class SpatialSelfAttention(nn.Module):
    def __init__(self, channels: int, num_heads: int = 4) -> None:
        super().__init__()
        self.norm = nn.GroupNorm(8, channels)
        self.qkv = nn.Conv2d(channels, channels * 3, kernel_size=1)
        self.proj = nn.Conv2d(channels, channels, kernel_size=1)
        self.num_heads = num_heads

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        b, c, h, w = x.shape
        norm_x = self.norm(x)
        qkv = self.qkv(norm_x).reshape(b, 3, self.num_heads, c // self.num_heads, h * w)
        q, k, v = qkv[:, 0], qkv[:, 1], qkv[:, 2] # [B, num_heads, head_dim, H*W]
        
        # Scaled dot-product attention
        q = q.transpose(-1, -2) # [B, num_heads, H*W, head_dim]
        k = k.transpose(-1, -2)
        v = v.transpose(-1, -2)
        
        attn = F.scaled_dot_product_attention(q, k, v)
        attn = attn.transpose(-1, -2).reshape(b, c, h, w)
        return x + self.proj(attn)


class DiffusionUNet(nn.Module):
    def __init__(self, in_channels: int = 3, out_channels: int = 3, base_dim: int = 64) -> None:
        super().__init__()
        time_dim = base_dim * 4
        self.time_embed = nn.Sequential(
            SinusoidalPositionalEmbedding(base_dim),
            nn.Linear(base_dim, time_dim),
            nn.SiLU(),
            nn.Linear(time_dim, time_dim),
        )

        # Encoder
        self.conv_in = nn.Conv2d(in_channels, base_dim, kernel_size=3, padding=1)
        self.down1 = ResBlockFiLM(base_dim, base_dim, time_dim)
        self.pool1 = nn.Conv2d(base_dim, base_dim, kernel_size=4, stride=2, padding=1)
        
        self.down2 = ResBlockFiLM(base_dim, base_dim * 2, time_dim)
        self.pool2 = nn.Conv2d(base_dim * 2, base_dim * 2, kernel_size=4, stride=2, padding=1)
        
        # Bottleneck with Self-Attention
        self.mid1 = ResBlockFiLM(base_dim * 2, base_dim * 2, time_dim)
        self.mid_attn = SpatialSelfAttention(base_dim * 2)
        self.mid2 = ResBlockFiLM(base_dim * 2, base_dim * 2, time_dim)

        # Decoder with Skip Connections
        self.up2 = ResBlockFiLM(base_dim * 4, base_dim, time_dim)
        self.up1 = ResBlockFiLM(base_dim * 2, base_dim, time_dim)
        self.conv_out = nn.Conv2d(base_dim, out_channels, kernel_size=3, padding=1)

    def forward(self, x: torch.Tensor, t: torch.Tensor) -> torch.Tensor:
        t_emb = self.time_embed(t)
        
        x_in = self.conv_in(x)
        d1 = self.down1(x_in, t_emb)
        p1 = self.pool1(d1)
        
        d2 = self.down2(p1, t_emb)
        p2 = self.pool2(d2)
        
        m = self.mid2(self.mid_attn(self.mid1(p2, t_emb)), t_emb)
        
        u2 = F.interpolate(m, scale_factor=2.0, mode="nearest")
        u2 = self.up2(torch.cat([u2, d2], dim=1), t_emb)
        
        u1 = F.interpolate(u2, scale_factor=2.0, mode="nearest")
        u1 = self.up1(torch.cat([u1, d1], dim=1), t_emb)
        
        return self.conv_out(u1)

Step 3: PyTorch Diffusion Training Step

from __future__ import annotations

import torch
import torch.nn.functional as F
from diffusion_engine import GaussianDiffusionEngine
from unet_architecture import DiffusionUNet


def train_step(
    model: DiffusionUNet,
    diffusion: GaussianDiffusionEngine,
    optimizer: torch.optim.Optimizer,
    images: torch.Tensor,
) -> float:
    model.train()
    optimizer.zero_grad(set_to_none=True)
    
    b, *_, device = *images.shape, images.device
    
    # 1. Sample uniform timesteps t
    t = torch.randint(0, diffusion.timesteps, (b,), device=device).long()
    
    # 2. Add Gaussian noise via forward marginal q(x_t | x_0)
    noise = torch.randn_like(images)
    x_t, ground_truth_noise = diffusion.q_sample(x_0=images, t=t, noise=noise)
    
    # 3. Model predicts the injected noise
    predicted_noise = model(x_t, t)
    
    # 4. Compute L2 Loss || epsilon - epsilon_theta ||^2
    loss = F.mse_loss(predicted_noise, ground_truth_noise)
    loss.backward()
    
    # Gradient clipping prevents divergence
    torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
    optimizer.step()
    
    return loss.item()

Empirical Benchmark Evaluation

Quantitative evaluation comparing generation quality and likelihood bounds across standardized benchmark datasets:

Troubleshooting Common Diffusion Faults

1. Contrast Burn and Boundary Saturation

• Symptom: Generated images exhibit harsh high-contrast lines, color banding, or dark edges.
• Root Cause: Training data was normalized to $[0,1]$ instead of $[-1,1]$, breaking zero-mean isotropic Gaussian assumptions at $t=T$.
• Remedy: Normalize datasets via transforms.Normalize((0.5,), (0.5,)) and clamp sampled outputs using torch.clamp(sample, -1.0, 1.0).

2. Early Loss Plateau with Structural Blurring

• Symptom: Loss plateaus at high values within the first $10\text{k}$ steps; outputs remain blurred.
• Root Cause: Timestep embedding projection weights disconnected or missing sinusoidal frequency mappings.
• Remedy: Verify sinusoidal positional encodings are injected into every residual block via FiLM modulation, and evaluate model sampling strictly using an Exponential Moving Average (EMA) shadow copy ($\mu =0.9999$).

3. Sampling Divergence Under High CFG Scales ($s>8.0$)

• Symptom: Samples generated with classifier-free guidance become oversaturated or degenerate into abstract high-frequency patterns.
• Root Cause: Unconditional and conditional vector variance disparity under aggressive linear extrapolation.
• Remedy: Apply CFG Rescaling (Lin et al., 2024): rescale guided noise standard deviation to match conditional vector norm, or reduce guidance scale to $s\in [5.0,7.0]$.

References

1. Ho, J., Jain, A., & Abbeel, P. (2020). Denoising Diffusion Probabilistic Models. Advances in Neural Information Processing Systems (NeurIPS 2020).
2. Sohl-Dickstein, J., Weiss, E., Khan, N., & Sompolinsky, H. (2015). Deep Unsupervised Learning using Nonequilibrium Thermodynamics. ICML 2015.
3. Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2021). Score-Based Generative Modeling through Stochastic Differential Equations. ICLR 2021.
4. Song, J., Meng, C., & Ermon, S. (2021). Denoising Diffusion Implicit Models. ICLR 2021.
5. Ho, J., & Salimans, T. (2022). Classifier-Free Diffusion Guidance. NeurIPS Workshop.
6. Karras, T., Aittala, M., Aila, T., & Laine, S. (2022). Elucidating the Design Space of Diffusion-Based Generative Models. NeurIPS 2022.