Machine Learning: k-Nearest Neighbors

These flashcards cover essential vocabulary and concepts related to k-nearest neighbors in machine learning, providing definitions and clarifying terms.

k-nearest neighbors (k-NN)

A nonparametric method used for classification or regression by finding the k nearest examples in the training data.

Parametric models

Models that summarize training data with a fixed set of parameters, independent of the number of training examples.

Nonparametric models

Models that rely on the data themselves and cannot be characterized by a bounded set of parameters.

Euclidean distance

The straight-line distance between two points in Euclidean space; it is useful when attributes are similar.

Manhattan distance

Also known as city block distance; it measures the distance between points in a grid-based path.

Curse of Dimensionality

A phenomenon where the distance between points increases in high-dimensional spaces, making nearest neighbors less meaningful.

k-dimensional tree (k-d tree)

A balanced binary tree structure that organizes data points in k dimensions, facilitating faster nearest neighbor searches.

Normalization

The process of scaling data to have a mean of zero and a standard deviation of one, often done using z-scores.

Instance-based learning

A type of learning where the model relies on specific instances of the training data rather than general parameters.

Time complexity of k-NN

The computational complexity of finding nearest neighbors, which is O(N) for datasets with N examples.