Data Mining Process and Iris Data Set

Chapter 1: Data Mining Process

  • Data Sources
    • Industrial Process Data: Captured at field and controller levels; also at operator and management levels.
    • Business Data: Includes applications like shopping basket analysis and customer segmentation.
    • Text Data: Derived from text documents, messages, and web documents.
    • Image Data: Collected from sources like smartphone cameras and satellite imagery.
    • Biomedical Data: Comprises genome data and lab results.

Definitions

  • Data Mining (DM): The process of extracting meaningful knowledge from vast datasets.
    • Knowledge: Defined as interesting patterns that are nontrivial, new, useful, or comprehensive.
  • Knowledge Discovery (KDD): A structured process that includes:
    • Preprocessing: Involves using a priori knowledge.
    • Knowledge Extraction: The actual mining phase.
    • Postprocessing: Evaluation of the mined knowledge.
  • Data Analytics (DA): The application of computing systems to analyze large datasets for decision support.
    • Intermediate Processes: DM, KDD, and DA are cyclical feedback processes requiring expert input.
  • Related Areas: Integrates fields such as statistics, signal theory, pattern recognition, computational intelligence, machine learning, and operations research.

Knowledge Discovery Process

  • Steps Include:
    • Clustering: Grouping similar items together.
    • Regression: Analyzing relationships among variables.
    • Documentation: Keeping records of the process.
    • Postprocessing: Final evaluations.
    • Interpretation and Evaluation: Making sense of the results.
    • Preparation, Planning: Organizing the approach to data collection.
    • Feature Generation: Creating new variables from existing data.
    • Data Selection: Choosing relevant data parts.
    • Transformation: Altering data for analysis.
    • Preprocessing: Data cleaning (filtering, cleaning) and ensures the data is prepared correctly.
    • Analysis and Visualization: Studying data patterns visually to extract insights.
    • Correlation & Forecasting: Understanding the interrelations between variables and predicting future trends.
    • Standardization & Classification: Ensuring data is comparable and assigning categories.

Chapter 2: Data and Relations

  • Example Topics:
    • Scales: Understanding measurement scales in data analysis.
    • Matrix Representation: Utilizing matrices to represent data hierarchically.
    • Relations: Exploring connections between different data points.
    • Distance Measures: Calculating dissimilarities between datasets.
    • Clustering & Proximity Measures: Techniques to identify the closeness of data points.
    • Sampling & Quantization: Methods for data collection and reducing information granularity.

Iris Data Set (Anderson 1935)

  • Overview:
    • Composed of n = 150 vectors with p = 4 dimensions concerning iris plants.
    • Classes:
    • Iris Setosa: 50 instances.
    • Iris Versicolor: 50 instances.
    • Iris Virginica: 50 instances.
    • Components:
    • Sepal Length
    • Sepal Width
    • Petal Length
    • Petal Width

Typical Questions

  • Which of the data might contain errors or false class assignments?
  • What is the error caused by rounding the data off to one decimal place?
  • What is the correlation between petal length and petal width?
  • Which pair of dimensions is correlated most?
  • None of the flowers in the data set has a sepal width of 1.8 centimeters. Which sepal length would we expect for a flower that did have 1.8 cm as its sepal width?
  • Which species would an Iris with a sepal width of 1.8 centimeters belong to?
  • Do the three species contain sub-species that can be identified from the data?