STAT243 Chapter 2 - Section 2.4 (1)

STAT243: Getting to Know Your Data

Section 2.4: Measuring Data Similarity and Dissimilarity

2.4.1 Introduction to Similarity and Dissimilarity

• In data mining, various applications utilize measurements of similarity and dissimilarity for clustering, outlier analysis, and nearest-neighbor classification.

• Similarities play a crucial role, for instance, in marketing by grouping customers based on shared characteristics (such as income, residence, etc.).

• Cluster: A collection of data objects that are similar to one another and dissimilar to those in other clusters.

2.4.2 Proximity Measures

• This section focuses on measures of proximity which quantify the similarity or dissimilarity of objects.

• Similarity Measure: Returns 0 if two objects are not alike; higher values indicate greater similarity.

• Dissimilarity Measure: Returns 0 if objects are identical; higher values indicate increased dissimilarity.

2.4.3 Data Matrix versus Dissimilarity Matrix

• Objects described by multiple variables lead to multidimensional data. If there are ( n ) objects and ( p ) attributes:

  • Objects: ( x_1 = x_{11}, x_{12}, , \ldots, x_{1p} ) for object 1, ( x_2 = x_{21}, x_{22}, , \ldots, x_{2p} ) for object 2, etc.

2.4.4 Data Matrix

• Data Matrix: Stores ( n ) data objects as a relational table (( n \times p ) matrix) with rows as objects and columns as attributes.

2.4.5 Dissimilarity Matrix

• Dissimilarity Matrix: Represents proximities for all pairs of ( n ) objects in an ( n \times n ) table.

  • Diagonal values are always 0 (i.e., dissimilarity of an object with itself).

  • The matrix is symmetric (i.e., ( d(i,j) = d(j,i) )).

2.4.6 Proximity Measures for Different Variables

• Types of Variables: Categorical (nominal), binary, ordinal, interval (numeric), and mixed types.

Proximity Measure for Nominal Variables

• Nominal variables can have multiple states (e.g., color).

• Dissimilarity Formula: ( d(i,j) = \frac{p - m}{p} ) where ( m ) is the number of matches and ( p ) is total variables.

Proximity Measure for Binary Variables

• Binary outcomes have two states: 0 (absent) and 1 (present).

• For symmetric variables: ( d(i,j) = \frac{r+s}{p} ), where ( r ) and ( s ) are counts based on conditions.

2.4.7 Proximity Measure for Numeric Variables

• Common distance measures include Euclidean, Manhattan, and Minkowski distances.

• Normalization may be required to give all variables equal weight.

2.4.8 Measures of Dissimilarity for Numeric Data

• Euclidean Distance: Defined as ( d(i,j) = \sqrt{\sum_{f=1}^{p} (x_{if} - x_{jf})^2} )

• Manhattan Distance: Calculated as ( d(i,j) = \sum_{f=1}^{p} |x_{if} - x_{jf}| )

• Minkowski Distance: Generalization with parameter ( h ): ( d(i,j) = (\sum_{f=1}^{p} |x_{if} - x_{jf}|^h)^{1/h} )

2.4.9 Proximity Measure for Ordinal Variables

• Ordinal variables carry a rank-order but unknown magnitudes.

• Convert ordinal values to ranks for dissimilarity assessment.

2.4.10 Dissimilarity for Mixed Types

• Mixed datasets require unified analysis combining variables of differing types.

• Compute dissimilarity using functions appropriate to each variable type (normalization is key).

2.4.11 Cosine Similarity

• Useful for term-frequency vector comparisons in text documents.

• Computes similarity based on angle between two vectors, focusing on non-zero elements to assess relevance.

• Cosine Similarity Formula: ( s(x,y) = \frac{x \cdot y}{||x|| ||y||} )

• High values indicate greater similarity.

Conclusion

• Understanding how to measure data similarity and dissimilarity is crucial for data analysis, particularly in clustering and classification tasks.