Data Sim and Clustering

Data Similarity and Clustering in Data Science (COMP5122M)

• Lecturer: Duygu Sarikaya

Course Overview

• Topics Covered:

  • Calculating Data Similarity

  • Data Distance Metrics

  • k-Means Clustering

  • Agglomerative Clustering

  • Hierarchical Clustering and Dendrograms

• Learning Objectives: Gain insights into data similarity, clustering methodologies, and hierarchical clustering.

What is Data Mining?

• Definition:

  • Data mining (or Knowledge Discovery in Data - KDD) is about uncovering patterns and valuable information from large datasets.

  • Source: IBM (https://www.ibm.com/topics/data-mining)

Data Mining vs. Machine Learning

• Data Mining:

  • Focuses on discovering existing patterns in data.

• Machine Learning:

  • Learns from past data to predict future outcomes.

• Source: Simplilearn (https://www.simplilearn.com/data-mining-vs-machine-learning-article)

Learning Types in Machine Learning

Supervised Learning

• Types:

  • Regression (Categorical Response)

  • Classification (Quantitative Response)

  • Nearest Neighbors

  • Logistic Regression

  • Goal: Creating a mapping function from inputs to outputs using example input/output pairs.

    • Each pair consists of an input vector and output value (label).

Unsupervised Learning

• Types:

  • Clustering

  • Dimensionality Reduction

• Goal: To identify patterns in unlabeled data (sometimes ignoring labels).

• Common Methods:

  • Principal Component Analysis (PCA)

Overview of Transformation of Dataset

• Purpose:

  • Create simpler representations of data for easier human or algorithm understanding.

• Examples:

  • Dimensionality Reduction (for visualization)

  • Topic extraction on text collections.

Clustering Techniques

• Definition:

  • Partitioning data into distinct groups of similar items.

    • Clusters are defined by high similarity within and low similarity between clusters.

• Examples of Application:

  • Organizing pictures by person or location.

Data Similarity

• Concept:

  • Measure the similarity (or distance) between objects represented as feature vectors.

• Application in Recommendations:

  • Used by platforms like Amazon, Netflix, and Spotify to suggest items based on user behavior.

Distance Metrics for Data Similarity

Euclidean Distance

• Formula Example:

  • Distance(A, B) = √[(X_B - X_A)² + (Y_B - Y_A)²]

• Dependence on Variables:

  • Adaptation for multiple variables potentially through extended dimension measures.

Other Distance Metrics

• Common Metrics:

  • Cosine Distance, Hamming, Manhattan, Minkowski, Chebyshev, Jaccard, Haversine, Sorensen-Dice

k-Means Clustering

• Overview:

  • A widely used clustering algorithm to find representative cluster centers.

• Process:

  • Assignment of data points to the nearest cluster center and updating the center positions iteratively.

• End Condition:

  • Stops when point assignments do not change.

Visualization in k-Means Clustering

• Cluster Representation:

  • Triangles for cluster centers, circles for data points, colors for cluster memberships.

Iterative Process Description

• Steps in Iteration:

  • Initialize cluster centers, assign points, and recompute centers until stability.

• Example Iterations:

  • Iterations demonstrate movement towards optimal cluster centers until stabilization.

Challenges in k-Means Clustering

• Cluster Shape Assumptions:

  • Assumes clusters are convex and equally shaped.

• Initialization Sensitivity:

  • Random initialization leads to variable outputs across runs.

• Effectiveness:

  • May fail on datasets with non-spherical shapes or differing cluster densities.

Hierarchical Clustering and Dendrograms

• Definition:

  • Builds a tree of clusters starting with each point as a separate cluster and progressively merging similar clusters.

• Visualization:

  • Dendrograms illustrate the merging process and cluster similarities across iterations.

• Limitations:

  • Ineffective for separating complex shapes under certain conditions.

Teaching References

• Books:

  • Data Science for Business by Foster Provost and Tom Fawcett

  • Introduction to Machine Learning with Python by Andreas Müller & Sarah Guido

Data Similarity and Clustering in Data Science (COMP5122M)

Lecturer: Duygu Sarikaya

Course Overview

• Topics Covered:

  • Calculating Data Similarity

  • Data Distance Metrics

  • k-Means Clustering

  • Agglomerative Clustering

  • Hierarchical Clustering and Dendrograms

• Learning Objectives: Insights into data similarity, clustering methodologies, hierarchical clustering.

Data Mining

• Definition: Uncovering patterns in large datasets.

• Difference from Machine Learning:

  • Data Mining: Discovering existing patterns.

  • Machine Learning: Predicting future outcomes based on past data.

Learning Types in Machine Learning

• Supervised Learning:

  • Types: Regression, Classification.

  • Nearest Neighbors

  • Logistic Regression

  • Goal: Mapping function from inputs to outputs using example pairs.

• Unsupervised Learning:

  • Types: Clustering, Dimensionality Reduction.

  • Goal: Identifying patterns in unlabeled data.

Clustering Techniques

• Definition: Partitioning data into groups of similar items.

• Examples: Organizing pictures.

Data Similarity

• Concept: Measure similarity/distance between feature vectors.

• Application: Recommendations on platforms like Amazon, Netflix.

Distance Metrics

• Euclidean Distance Formula: Distance(A, B) = √[(X_B - X_A)² + (Y_B - Y_A)²].

• Other Metrics: Cosine, Hamming, Manhattan, Minkowski, Chebyshev, Jaccard, Haversine, Sorensen-Dice.

k-Means Clustering

• Overview: Assigns points to nearest cluster centers; iterates until stable.

• Challenges: Assumes convex shapes; sensitive to initialization; may fail with non-spherical shapes.

Hierarchical Clustering

• Definition: Builds a tree of clusters; visualized with dendrograms.

• Limitations: Complex shapes can be poorly separated.

Teaching References

• Books:

  • Data Science for Business by Foster Provost and Tom Fawcett

  • Introduction to Machine Learning with Python by Andreas Müller & Sarah Guido