Unsupervised Learning

Labeled observations

Each observation is a tuple (x,y) of feature vector x and output label y which are related according to an unknown function f(x) = y

Supervised Learning

* Labeled observations: Each observation is a tuple (x,y) of feature vector x and output label y which are related according to an unknown function f(x) = y
* During training: Learn the relationship between x and y i.e., find a function (or model) ℎ(x) that best fits the observations
* Goal: Learned model accurately predicts the output label of a previously unseen, test feature input (generalization)
* Labels : ‘Teachers’ during training, and ‘validator’ of results during testing

Unsupervised Learning

Unlabeled data set of feature vectors

What can unsupervised learning deduce from classification data?

find sub-groups (or clusters) among observations with similar traits (clustering)

What can unsupervised learning deduce from regression data?

find patterns within feature vector to identify a lower dimensional representation (dimensionality reduction)

Challenges of unsupervised learning

* No simple goal as in supervised learning
* Validation of results is subjective
* Often more used in exploratory data analysis

Advantages of unsupervised learning

* Labeled data expensive and difficult to collect; unlabeled data cheap and abundant
* Compressed representation saves on storage and computation
* Reduce noise, irrelevant attributes in high dimensional data
* Pre-processing step for supervised learning

What is clustering?

What are clusters formed based on?

Clustering is subjective: clusters are formed based on a user-specified measure of similarity that depends on domain knowledge.

what tyoe of classification is clustering

unsupervised learning - since labels are derived only from the observations

Normalization of Feature Vectors

so that no data is more weighted than the others. All data is between 0 and 1 - drawback is that its sensitive to outliers

Z-score standardization

all feature attributes have mean 0 and standard deviation 1. - drawback - not bounded range

Types of Clustering Algorithms

* Partitional
* Hierarchical
* Model-Based

Partitional Clustering

* Generates a single partition of the data to recover natural clusters
* Input: Feature vectors
* Examples: K-means, K-medoids
* Goal: assign N observations into K (K

Hierarchical Clustering

* Generates a sequence of nested partitions
* Input: Distance Matrix
* Example: agglomerative clustering, divisive clustering

Model-Based Clustering

* Assumes that data is generated i.i.d. from a mixture of distributions, each of which determines a different cluster
* Example: Expectation-Maximization (EM)

Measure of intra-cluster similarity

Dissimilarity within a clustering structure C