Chapter 1-8: Deep Learning & Imitation Learning - Vocabulary Flashcards

Vocabulary flashcards covering core concepts from the lecture notes on supervised/unsupervised/self-supervised learning, reinforcement learning basics, neural networks, CNNs, imitation learning, and related techniques.

Supervised learning

Training with input-output pairs (x, y) to learn a function f that maps inputs to labels.

Unsupervised learning

Learning from data without labeled outputs; the model discovers structure or representations.

Self-supervised learning

A form of learning where labels are generated from the data itself (pretext tasks); popular in language models.

Reinforcement learning

Agent learns by interacting with an environment, taking actions to maximize cumulative rewards.

Classification

A supervised learning task where inputs are mapped to discrete labels, e.g., iris species.

Iris dataset

Classic four-feature dataset used for plant species classification: sepal/petal length and width.

Generalization

Model performance on unseen data beyond the training set.

IID (independent and identically distributed)

Assumes training and test samples come from the same distribution and are independent.

Training/testing split

Dividing data into training and testing sets (e.g., 80/20) to assess generalization.

Gradient descent

Optimization method that updates parameters in the direction of steepest descent of the loss.

Backpropagation

Algorithm to compute gradients across neural network layers using the chain rule.

Activation function

Nonlinear function applied to neuron outputs to enable nonlinear modeling.

ReLU (Rectified Linear Unit)

Activation f(x) = max(0, x); fast to compute and has simple gradients.

Sigmoid

S-shaped activation mapping inputs to (0, 1); commonly used for probabilities in binary tasks.

Softmax

Activation that converts logits to a probability distribution over multiple classes.

Neural network

Computational model with layers of neurons; learns via weights and biases.

Deep neural network

Neural network with multiple hidden layers; depth provides greater representational power.

Convolutional neural network (CNN)

Neural network using convolutional layers to capture spatial structure in images.

Spatial locality

CNNs capture local spatial relationships via small receptive fields.

Translation invariance / equivariance

Ability to detect patterns regardless of position due to sliding filters.

Parameter sharing

Same filter parameters used across different image locations, reducing parameters.

Behavior cloning

Imitation-learning approach that mimics teacher demonstrations via supervised learning.

Imitation learning

Learning a policy from expert demonstrations (state-action pairs) to imitate behavior.

Demonstrations

Recorded state-action sequences provided by a teacher or expert.

Dataset Aggregation (DAG)

Iterative imitation-learning method that interleaves teacher and student actions and aggregates data to reduce covariate shift.

Multimodality

Situations where multiple actions may be appropriate; requires stochastic policies to model multiple modes.

Diffusion policy

A method using diffusion models to sample from a multi-modal action distribution.

Kinesthetic teaching

Physically guiding a robot by hand to demonstrate movements.

Teleoperation

Remote control of a robot to collect demonstrations via a human operator.

Feature maps

Intermediate representations produced by CNN layers that reveal detected features (e.g., edges).