L22_More Generative Deep Learning

Flashcards covering generative deep learning methods, including autoencoders, GANs, and diffusion models.

Generative Deep Learning

Deep learning methods that learn to represent and generate realistic data from random samples. Think of it like a digital artist learning to paint by analyzing countless artworks. Key methods include Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), and Diffusion Models.

Autoencoder

A neural network trained to copy its input to its output. It works by compressing the input into a lower-dimensional code (latent space) and then reconstructing the output from this code. It's like zipping a file to make it smaller and then unzipping it back to its original size.

Latent Space Sampling

The process of generating new data by randomly selecting values from the latent space of a generative model. Imagine the latent space as a palette of colors; sampling is like picking random colors to create a new painting.

Improving the Autoencoder

Techniques used to ensure that the latent space of an autoencoder is well-structured and suitable for generating realistic outputs. This involves imposing constraints on the latent space during training, such as encouraging continuity and completeness.

Learning Distributions

A technique where models predict the parameters (e.g., mean and variance) of a probability distribution rather than a single value. This allows the model to capture the uncertainty and variability in the data, enabling it to generate more realistic and diverse outputs.

Variational Autoencoder (VAE)

An autoencoder that models distributions in the latent space. The encoder maps input data to a probability distribution (mean and variance), and the decoder reconstructs the data from samples drawn from this distribution. p(z|x). Think of it like learning the 'style' of an image, rather than just memorizing the image itself.

VAE Loss Function

Combines a reconstruction loss (e.g., Mean Squared Error) that measures how well the decoder reconstructs the input, and a regularization term (Kullback-Leibler divergence) that measures the difference between the latent distribution and a standard normal distribution. L = L{reconstruction} + L{KL}. This ensures the latent space is well-behaved and suitable for sampling.

VAE Latent Space

A continuous and structured latent space that allows for meaningful interpolation and manipulation. Sampling along different dimensions of the latent space can combine or remove concepts, allowing for creative control over the generated output. It's like having knobs to control different aspects of an image.

Generative Adversarial Network (GAN)

A framework consisting of two neural networks: a generator that creates images from random noise, and a discriminator that tries to distinguish between real and generated images. The two networks are trained in an adversarial manner, with the generator trying to fool the discriminator and the discriminator trying to catch the generator. It's like a game of cat and mouse, where the generator and discriminator are constantly trying to outsmart each other.

Diffusion Models

A class of generative models that gradually add random noise to an image over multiple steps (forward diffusion process) and then learn to reverse this process to generate images from noise (reverse diffusion process).

Diffusion Model as VAE

A diffusion model can be viewed as a variational autoencoder where the encoder process is simply adding noise. This provides a theoretical framework for understanding diffusion models in terms of probabilistic inference and variational inference.

Generating Images with Diffusion Models

The process of generating images with diffusion models involves starting with an image of random noise and then iteratively denoising it using the learned reverse diffusion process. Each step refines the image, gradually converging towards a realistic image.

α¯t (Alpha bar)

Represents the proportion of the original image that remains after t steps of the diffusion process. As t increases, α¯t decreases, indicating that more noise has been added to the image.

Reverse Decoder

A neural network that approximates the reverse diffusion process, i.e., the process of removing noise from an image. This is typically implemented using a U-Net architecture, which is well-suited for processing images at multiple scales.

Generating New Samples (Diffusion Models)

Generating new samples with diffusion models involves running the reverse decoder sequentially to remove noise from random pixel values drawn from a normal distribution. This process gradually transforms random noise into a coherent and realistic image.

Latent Diffusion Models

A variant of diffusion models that operates in the latent space of a variational autoencoder rather than directly in pixel space. This can improve the efficiency and quality of the generated images by reducing the dimensionality of the data.

Conditional Generation (Guided Diffusion)

A technique for guiding the image generation process based on additional input data, such as a class label or text prompt. This allows for more control over the generated output and enables the creation of images that satisfy specific criteria.

Guided Diffusion Implementation

A common approach to guided diffusion involves encoding the input prompt (e.g., using a text encoder) and concatenating it with the input to the denoising network. Cross-attention layers are often used to allow the denoising network to attend to the relevant parts of the input prompt.