1/34
Vocabulary practice flashcards reviewing core deep learning concepts including model capacity, bias-variance trade-off, optimization via SGD, feedforward nets, activation and loss functions, CNN operations, LSTM gates, and RNN design patterns.
Capacity
The range of functions that a model can approximate, reflecting its ability to learn different patterns from data.
Width
The number of nodes contained within a single layer of a neural network model.
Depth
The total number of layers present in a neural network model.
Overfitting
A condition where a model performs very well on training data but poorly on unseen test data by memorizing training data rather than generalizing patterns.
Underfitting
A condition where a model fails to learn patterns in the training data properly and cannot generalize well to new data, resulting from an overly simple model.
Reducible Errors
Errors in machine learning that can be minimized to improve model accuracy; they are categorized into bias and variance.
Irreducible Errors
Inherent model errors that cannot be reduced because they stem from unknown, uncontrollable variables.
Bias
The difference between a model's predicted values and actual values, representing an algorithm's inability to capture true underlying data relationships due to excessive assumptions.
Variance
The amount of variation observed in a model's predictions when trained across different training datasets.
Bias-Variance Trade-off
The process of balancing bias and variance errors to find an optimal sweet spot that minimizes both underfitting and overfitting in supervised learning.
Stochastic Gradient Descent (SGD)
An optimization algorithm that minimizes the loss function by iteratively updating model parameters using one randomly selected training example at a time.
Deep Feed Forward Network
An artificial neural network (also called a Multilayer Perceptron) where data strictly moves forward from the input layer through one or more hidden layers to the output layer without looping backwards.
Rectified Linear Unit (ReLU)
A non-linear activation function defined as f(x)=max(0,x), which outputs the input directly if positive and outputs zero if negative.
Leaky ReLU
A non-linear activation function variant that allows a small, non-zero negative value when the input is negative instead of returning zero.
Parametric ReLU (PReLU)
An activation function that learns the slope parameter for negative input values during training rather than keeping it fixed.
Exponential Linear Unit (ELU)
A non-linear activation function that generates smooth, continuous negative values for inputs below zero.
Scaled Exponential Linear Unit (SELU)
A non-linear activation function designed to induce self-normalizing properties and maintain stable values across deep network layers.
Loss Function
A function that measures the prediction error for a single individual data point.
Cost Function
A function that calculates the aggregate prediction error across an entire dataset by combining all individual loss values.
Mean Squared Error (MSE)
A regression loss function that squares the differences between actual and predicted values and averages them, making it sensitive to outliers.
Mean Absolute Error (MAE)
A regression loss function that calculates the average of the absolute differences between actual and predicted values, offering greater robustness to outliers than MSE.
Huber Loss
A regression loss function combining properties of MSE and MAE to achieve outlier robustness while maintaining smoothness.
Binary Cross-Entropy
A classification loss function used for binary (two-class) problems, typically used in conjunction with a Sigmoid activation in the final neuron.
Categorical Cross-Entropy
A multi-class classification loss function used with one-hot encoded target vectors, typically paired with a Softmax activation in the output layer.
Sparse Categorical Cross-Entropy
A multi-class classification loss function that accepts integer labels instead of one-hot encoded vectors, providing faster computation.
Convolutional Neural Network (CNN)
A specialized deep learning architecture designed for image processing that automatically extracts spatial features such as edges, textures, and objects.
Transposed Convolution
A convolutional operation used for up-sampling that converts smaller feature maps into larger ones to reconstruct or generate images.
Dilated Convolution
Also called Atrous Convolution; an operation that expands the receptive field by skipping pixels between filter elements without increasing parameter count.
Long Short-Term Memory (LSTM)
A recurrent neural network variant engineered with gating mechanisms to retain long-term dependencies and resolve the vanishing gradient problem.
Cell State
The memory pathway in an LSTM network that carries and preserves long-term information across time steps.
Forget Gate
An LSTM gating component that employs a Sigmoid function to output values between 0 and 1, determining which old information to discard from the cell state.
Input Gate
An LSTM gating component utilizing Sigmoid and Tanh activations to decide which new information should be stored in the cell state.
Output Gate
An LSTM gating component that regulates which parts of the cell state are emitted as the final hidden state or output.
Bidirectional RNN
A recurrent architecture that concurrently deploys forward and backward RNN layers to process sequential data using both past and future contextual information.
Sequence-to-Sequence (Seq2Seq) RNN
An RNN framework featuring an encoder that compresses an input sequence into a context vector and a decoder that generates a new sequence from that vector.