CS3001 Section 2 - Classification

Step in the K-Nearest Neighbour algorithm

1. Calculate Euclidean distance from the NEW point to EVERY training point.

2. Sort all training points by distance. Select the K CLOSEST ones.

3. Majority vote among those K neighbours — that class is the prediction.

What is a memory tip for choosing K in K-Nearest Neighbour?

Use an odd K to avoid ties, such as K=3 or K=5.

What is the process for classifying new data using Decision Trees?

Start at the root node, check the condition, and follow the yes/no branches until reaching a leaf node.

Advantages of Decision Trees

Provide human-readable rules that are easy to interpret.

No need to scale or normalise features

What is a disadvantage of Decision Trees?

Prone to overfitting without pruning

Only creates axis-parallel (orthogonal) splits

Disadvantages for K-Nearest Neighbour

Slow at prediction — compares to ALL training points

Very sensitive to choice of K and distance metric

Must store all training data in memory

Advantage of K-Nearest Neighbours

No assumptions about data distribution

Easy to understand and interpret

No model training required