Cogs 118B: Intro to Machine Learning - Google Docs

What is the primary objective of Machine Learning as discussed in the context of Cogs 118B?

To study and construct algorithms that can learn from and make predictions on data by building a model from observations.

How is Supervised Learning characterized in introductory machine learning?

A learning paradigm where the model is trained on a labeled dataset, meaning each training example is paired with an output label.

Define Unsupervised Learning.

A type of machine learning that looks for previously unknown patterns in a dataset without pre-existing labels.

What is a 'feature' in a machine learning dataset?

An individual measurable property or characteristic of a phenomenon being observed, often represented as a column in a data matrix.

What is the role of the 'label' or 'target' variable?

The ground truth or outcome that the machine learning model is tasked with predicting based on the input features.

Define the Hypothesis Function $h(x)$.

The mathematical function that a learning algorithm uses to find the mapping from the input features to the output variable.

What is the goal of a Linear Regression model?

To approximate the relationship between dependent and independent variables using a linear equation, usually to predict continuous values.

What is the purpose of the Cost Function in machine learning?

To quantify the difference between the model's predictions and the actual target values, typically used to measure model performance.

Write the formula for the Mean Squared Error (MSE) cost function.

$$J(\theta) = \frac{1}{2m} \sum_{i=1}^{m} (h_{\theta}(x^{(i)}) - y^{(i)})^2$$

What is Gradient Descent?

A first-order iterative optimization algorithm for finding the local minimum of a differentiable function by moving proportional to the negative of the gradient.

In Gradient Descent, what does the parameter $\alpha$ represent?

The learning rate, which determines the magnitude of the steps taken toward the minimum of the cost function.

What distinguishes Logistic Regression from Linear Regression?

Logistic Regression is used for classification tasks producing discrete outputs, whereas Linear Regression is used for regression tasks producing continuous outputs.

What is the Sigmoid Function's equation?

$$g(z) = \frac{1}{1 + e^{-z}}$$

Define Overfitting.

A phenomenon where a model fits the training data too well, including the noise, resulting in poor generalization to new, unseen data.

Define Underfitting.

A state where a model is unable to capture the underlying trend of the data, typically due to the model being too simple for the complexity of the data.

Describe the Bias-Variance tradeoff.

The fundamental trade-off between the error from the model's assumptions (Bias) and the error from sensitivity to training data noise (Variance).

What is the specific purpose of Regularization?

To prevent overfitting by adding a penalty term for large parameter values to the cost function, encouraging simpler models.

What is L1 Regularization, also known as Lasso?

A technique that adds a penalty proportional to the sum of the absolute values of the weights: $\lambda \sum_{j=1}^{n} |\theta_j|$.

What is L2 Regularization, also known as Ridge?

A technique that adds a penalty proportional to the sum of the squares of the weights: $\lambda \sum_{j=1}^{n} \theta_j^2$.

In ML evaluation, what is a Training Set?

A subset of the original data used to train the model and adjust its internal parameters.

In ML evaluation, what is a Test Set?

A separate portion of the dataset used only at the end of the development process to provide an unbiased evaluation of the final model.

What is the K-Nearest Neighbors (KNN) algorithm?

A simple classification algorithm that assigns a label to a data point based on the majority label of its $k$ closest points in the feature space.

Define a Support Vector Machine (SVM).

A classification model that finds the hyperplane which maximizes the separation (margin) between two classes of data.

What is the formula for Precision in a classification task?

$\text{Precision} = \frac{TP}{TP + FP}$ where $TP$ is True Positives and $FP$ is False Positives.

What is the formula for Recall (Sensitivity) in a classification task?

$\text{Recall} = \frac{TP}{TP + FN}$ where $TP$ is True Positives and $FN$ is False Negatives.