Comprehensive Guide to K-Nearest Neighbors (KNN) Algorithm in Machine Learning

What is the primary goal of the KNN algorithm?

To classify or predict the label of a data point based on the majority label of its k nearest neighbors.

What type of learning is KNN classified under?

KNN is a supervised learning algorithm.

What does K represent in KNN?

K represents the number of nearest neighbors considered in the prediction process.

What is the key idea behind KNN?

Similar data points should have similar labels.

What distance metric is commonly used in KNN?

Euclidean distance.

What is the first step in the KNN algorithm?

Calculate the distance of the test point from all training points.

How does KNN classify a new data point?

By using the majority vote of the k nearest neighbors' labels.

What is a potential drawback of KNN in imbalanced datasets?

The majority class can dominate predictions, leading to poor detection of the minority class.

What is the accuracy formula in KNN?

Accuracy = number of correct predictions / total number of predictions.

What happens if K is set to 1 in KNN?

The prediction is based solely on the label of the nearest neighbor.

What is a consequence of high accuracy in imbalanced datasets?

It can be misleading, as the model may simply predict the majority class.

What is the curse of dimensionality in KNN?

As the number of features increases, data points become sparse and distance metrics become less meaningful.

What are some advantages of KNN?

Effective for non-linear data, ease of implementation, and versatility in classification and regression tasks.

What are some limitations of KNN?

Computational cost, sensitivity to noise, and performance degradation with increased dimensionality.

What is distance-weighted voting in KNN?

A method to give more influence to closer neighbors when predicting the label of a new data point.

How can one improve KNN's performance on imbalanced datasets?

By using distance-weighted voting, choosing a smaller k, or applying resampling techniques.

What is hyperparameter tuning in the context of KNN?

The process of selecting the best value for k to optimize the model's performance.

What is the role of labeled examples in KNN?

Labeled examples are used to determine the class of the test point based on proximity.

What is the impact of outliers on KNN predictions?

Outliers can significantly influence the classification or regression output.

What does KNN require for its prediction step?

It requires computing the distance from the test point to all training points.

What is the majority vote rule in KNN?

The class with the most votes among the k nearest neighbors is assigned to the new data point.

What is the significance of the training phase in KNN?

Training is fast as it involves simply storing the labeled instances.

What is the effect of increasing k in KNN?

Increasing k can lead to smoother decision boundaries but may also introduce bias towards the majority class.

What is the difference between model parameters and hyperparameters?

Model parameters are learned from data, while hyperparameters are set before training and control the learning process.

What is an example application of KNN?

Movie recommendation systems and detecting phishing emails.

What is the main question KNN seeks to answer?

Which known data point does the unknown data point most resemble?

What is the computational complexity of KNN?

O(dN) for N training examples in d dimensions.

What is a potential solution to the class imbalance problem in KNN?

Using resampling techniques like SMOTE or undersampling.