3.5 Cluster Analysis

Section 3.5 of Exam MAS-II

k-Means Clustering: Definition

k-means clustering is a clustering algorithm that aims to partition the observations in a dataset into a ____________ number of clusters. The algorithm aims to find the cluster assignments with the lowest total _________ _________.

k-means clustering is a clustering algorithm that aims to partition the observations in a dataset into a pre-specified number of clusters. The algorithm aims to find the cluster assignments with the lowest total within-cluster variation.

k-Means Clustering: Algorithm

1. Randomly assign a cluster to each observation. This serves as the initial cluster assignments.

2. Calculate the centroid of each cluster.

3. For each observation, identify the closest centroid and reassign to that cluster.

4. Repeat steps 2 and 3 until the cluster assignments stop changing.

Within-Cluster Variation Formula (and total)

For every observation in a cluster, sum the squared distance between the observation and the centroid. Multiply the total by 2 to account for all pair-wise observations

Total: Add the within-cluster variation for all clusters

Hierarchical Clustering: Definition

Hierarchical clustering is a clustering method that results in a ________ representation of the observations called a __________.

Hierarchical clustering is a clustering method that results in a tree-like representation of the observations called a dendrogram.

Hierarchical Clustering: Algorithm

Hierarchical Clustering: Linkages

• Complete: The largest dissimilarity is selected as the inter-cluster dissimilarity.

• Single: The smallest dissimilarity is selected as the inter-cluster dissimilarity.

• Average: The arithmetic mean is selected as the inter-cluster dissimilarity.

• Centroid: The dissimilarity between the cluster centroids is selected as the inter-cluster dissimilarity.

Clustering: Other Key Ideas

Key difference between k-means clustering and hierarchical clustering:

• For k-means clustering, the algorithm needs to be ______ ______ _____ for a specific value of k, as well as for different values of k.

• For hierarchical clustering, the algorithm only needs to be performed ____ for any number of clusters.

Key difference between k-means clustering and hierarchical clustering:

• For k-means clustering, the algorithm needs to be repeated multiple times for a specific value of k, as well as for different values of k.

• For hierarchical clustering, the algorithm only needs to be performed once for any number of clusters.

Clustering: Other Key Ideas

The result of clustering depends on many considerations, such as:

• Choice of ______ in k-means clustering

• Choice of number of _______, _______, and ________ _______ in hierarchical clustering

• Choice to ________ variables

The result of clustering depends on many considerations, such as:

• Choice of k in k-means clustering

• Choice of number of clusters, linkage, and dissimilarity measure in hierarchical clustering

• Choice to standardize variables