Ch7: Advanced Learning Models

What are the 3 components of an autoencoder?

Encoder, Code (latent space), Decoder

Is autoencoder supervised or unsupervised?

Unsupervised (self-supervised because input = output)

Why can't an autoencoder trained on digits compress faces?

Autoencoders are data-specific, they only learn features from training data.

Is autoencoder lossless or lossy?

Lossy, output is not exactly the same as input

What loss functions are used for autoencoders?

MSE (any value range) or binary crossentropy (values between 0 and 1)

Name 3 applications of autoencoders

Dimensionality reduction, image denoising, anomaly detection

What is the main difference between VAE and regular autoencoder?

VAE encoder outputs a probability distribution (mean + std dev), not a single code

What is the reparameterization trick?

sample = mean + (std dev × noise), allows backpropagation through random sampling

What are the two terms in VAE loss function?

Reconstruction loss + Regularization term (KL divergence)

What is one advantage of VAE over regular autoencoder?

Can generate NEW data (not just reconstruct)

What is one disadvantage of VAE compared to GAN?

Produces blurrier images

How do VAEs help with anomaly detection?

Anomalies have high reconstruction error because they don't fit the learned distribution

What two networks compete in a GAN?

Generator and Discriminator

What is generator's goal?

Create fake data that fools discriminator

What is discriminator's goal?

Correctly identify real vs fake data

what can the generator do?

Reproduce the true data distribution perfectly

Name 4 applications of generative AI by modality

Language (LLMs), Audio (music), Visual (images/video), Synthetic data

What is synthetic data used for?

Training AI when real data doesn't exist, is restricted, or can't cover edge cases

Name ethical concerns of generative AI

Bias/fairness, privacy, deepfakes, accountability, IP rights, authenticity

Why is bias a problem?

AI trained on biased data produces discriminatory content (race, gender, etc.)