Fundamentals and Pre-processing in Data Mining

Introduction to Data Mining

Data mining is defined as a set of methods applied to large and complex databases. The primary objectives of these methods are to eliminate randomness and discover hidden patterns within the data.

Datasets

A dataset is a collection of related sets of information composed of separate elements that can be manipulated as a single unit by a computer. Structurally, a dataset takes the form of:

• Instances: Individual records or examples in the dataset.
• Attributes: The specific features or characteristics of the instances (also referred to as features).
• Class: The label or category assigned to an instance (also referred to as the target variable).

Nature of Datasets

Datasets are categorized based on the nature of the data they contain:

Quantitative Data

This involves measurements or counts recorded as numerical values. Examples include:

• Height
• Temperature
• The number of Red M&M’s in a bag

Qualitative Data

This consists of groups, types, or categories represented by names, symbols, or number codes. Examples include hair color, religion, political party, and profession. Qualitative data is further subdivided into:

• Ordinal Data: This data possesses a natural ordering. Examples include shirt sizes (e.g., $S$, $M$, $L$, $XL$).
• Nominal Data: These are simply names of categories without an inherent order. Examples include marital status, gender, or the color of M&M’s in a bag.

Sources of Free Datasets Online

Several platforms provide free datasets for research and project work:

• FiveThirtyEight
• BuzzFeed
• Google Cloud Platform
• UCI Machine Learning Repository
• Amazon Web Services (AWS)
• Data.world
• Academic Torrents
• Wikipedia
• Kaggle
• Data.gov
• Dataquest (blog/free-datasets-for-projects)

Data Mining and Ethics

Discrimination

Data mining is frequently utilized to discriminate in processes such as determining loan eligibility or selecting recipients for special offers. Certain forms of discrimination—including racial, sexual, and religious—are not only unethical but illegal. While using sexual and racial information is considered ethical in medical diagnostic contexts, using the same information to mine loan payment behavior is unethical.

Privacy and Transparency

• Anonymization: The process of removing identifying information from data so that the original source or individual cannot be identified.
• Re-identification: The risk of identifying individuals from anonymized datasets.
• Policy: When information is collected, individuals should be informed straightforwardly in plain language rather than legalistic small print.

Data Pre-processing

Pre-processing is a critical stage that involves several steps to prepare data for mining:

1. Data Cleaning: Removing noise and handling missing values.
2. Data Integration: Combining data from multiple heterogeneous sources (databases, data cubes, or files).
3. Data Reduction: Reducing the volume but producing the same or similar analytical results.
4. Data Transformation: Converting data into forms suitable for mining.
5. Data Mining: The actual core process of identifying patterns.
6. Pattern Evaluation: Using summarization and visualization to make data understandable.
7. Knowledge Representation: Using tools to represent the mined data.

Data Cleaning Details

Data cleaning involves the removal of noisy or incomplete data.

• Handling Missing Data: This can be achieved by ignoring the tuple, filling the missing value manually, using measures of central tendency, or filling in the most probable value.
• Noise Removal: Noise refers to random errors. Methods to remove noise include:
  • Binning: Sorting values into "buckets" or bins and smoothing them by consulting neighboring values.
  • Identifying Outliers: Locating data points on a graph that are significantly larger or smaller than the next nearest data point.

Data Transformation Strategies

• Smoothing: Removing noise using clustering or regression.
• Aggregation: Applying summary operations to the data.
• Normalization: Scaling data to fall within a specific range (e.g., $0.0$ to $1.0$) using methods like min-max normalization.
• Discretization: Replacing raw numeric values with intervals (e.g., converting specific ages into age groups).

Data Reduction Categories

• Dimensionality Reduction: Reducing the number of attributes.
• Numerosity Reduction: Replacing the original data volume with smaller forms of representation.
• Data Compression: Creating a compressed representation of the original data.

Learning Types in Data Mining

Classification Learning

The system is presented with classified examples and is expected to learn a method for classifying unseen examples.

Association Learning

The system seeks any association among features, rather than just those predicting a specific class value.

Numeric Prediction

The predicted outcome is a numeric quantity rather than a discrete class.

Learning by Concept Description

• Supervised Learning: This is like a teacher supervising the process. It involves Annotation, which is the process of labeling or naming the class of a given instance (e.g., labeling an animal as a "Mammal" or "Reptile").
• Unsupervised Learning: Algorithms are left to their own devices to discover and present interesting structures within the data. An example is Clustering, which groups similar items (e.g., grouping Fish, Insects, and Birds based on shared characteristics) without prior labeling.

Procedural Examples and Exercises

Binning and Smoothing Application

Given the data set: $2, 8, 14, 15, 16, 19, 21, 23, 36, 38, 40, 43, 44, 50, 52$

1. Partition into 3 bins: Each bin contains $5$ instances.
2. Smoothing by bin means: Replace every value in a bin with the average value of that bin.
3. Smoothing by bin-boundaries: Replace every value in the bin with the closest boundary value ($min$ or $max$ of the bin).

Case Study: Crime Dataset Cleaning

A raw dataset containing records of "Carnapping" and "Unjust Vexation" with attributes: Crime, Date, Time, Location, Victim.

• Step 1: Date Consistency: Standardize inconsistent date formats (e.g., changing "December 15, 2017" to "$12/15/2017$").
• Step 2: Missing Values: Fill in missing locations or times using the most probable value (e.g., if most carnappings occurred in "Binondo, Manila," use that to fill blanks for identical crimes in that timeframe).
• Step 3: Outlier Removal: If the mining goal is specific to carnapping, instances of "Unjust Vexation" are considered outliers and should be removed.
• Step 4: Ethics: Remove the "Victim" names to satisfy privacy and anonymization requirements.
• Step 5: Integration: Search for external data (e.g., Accuweather) to add a "Weather" attribute based on the existing Date and Location.
• Step 6: Reduction: Simplify location descriptions (e.g., reducing "Masangkay St., Binondo, Manila" simply to "Binondo, Manila").

Case Study: Weather and Play (Discretization)

In a dataset determining if one should "Play" based on Outlook, Temp, Humidity, and Windy:

• The system may split on Humidity halfway between the values of $70$ and $80$, setting a threshold at $75$.
• Instances with Humidity $\le 75$ might result in "Yes," while Humidity $> 75$ might result in "No."

Pattern Evaluation and Knowledge Representation

Pattern evaluation involves using measures to identify truly interesting patterns.

• Visualization: Tools are used to represent mined data and make it understandable to the user.
• Error Analysis: Evaluation involves tracking Training Error vs. Validation Error over multiple Epochs.
• Confusion Matrix: A table used to evaluate the accuracy of a classification model, comparing Actual classes (e.g., Spam vs. Non-spam) against Predicted classes.
  • Example: Actual Spam identified correctly as Spam ($600$), while $300$ Spam instances were incorrectly predicted as Non-spam.