Lecture 5 – Logistic Regression, K-Nearest Neighbors and Support Vector Machine

These flashcards cover key vocabulary and important concepts related to logistic regression, k-nearest neighbors, and support vector machines in machine learning.

Logistic Regression

A binary classifier used for linearly separable data to predict probabilities that lie between 0 and 1.

Sigmoid Function

An activation function used in logistic regression defined as σ(x) = 1 / (1 + e^(-x)).

Binary Classifier

A classification algorithm that distinguishes between two classes.

Linear Regression

A regression algorithm that predicts values and can output values greater than 1 or less than 0.

K-Nearest Neighbors (KNN)

An algorithm that classifies a data point based on its nearest neighbors, without building a decision boundary.

Euclidean Distance

A metric used to calculate the distance between two points in a multidimensional space.

Manhattan Distance

A metric that computes the distance between points in a grid-based path.

Hyperplane

A decision boundary that separates different classes in a multi-dimensional space.

Support Vector Machine (SVM)

A classification method that maximizes the margin between two classes.

Margin

The distance between the separating hyperplane and the nearest data points from either class.

Support Vectors

The data points that lie closest to the decision boundary in SVM.

Classification Error

The rate at which an algorithm incorrectly predicts the class labels of data points.

Threshold

A value used in logistic regression to determine the class assignment based on predicted probabilities.

Probabilities

Values that represent the likelihood of class membership between 0 and 1.

Activation Function

A function used in neural networks to introduce non-linearity into the model.

Kernel Trick

A method used in SVM to allow the algorithm to fit the hyperplane in a transformed feature space.

Optimization

The process of adjusting the parameters of a model to minimize error.

Dependent Variable

A variable that is being predicted or explained in a regression or classification model.

Independent Variable

A variable that is used as a predictor or input in a model.

Fit

To train a model on a given dataset.

Class Labels

The categorical outcomes that a classification model attempts to predict.