ML: Introduction to Knowledge Discovery

Erasmus Mundus Joint Master in Artificial Intelligence for Sustainable Societies (AISS)

Topics on Machine Learning and its Applications

Introduction to Knowledge Discovery

  • Introduction to Knowledge Discovery

  • Knowledge Discovery Definitions

    • Knowledge Discovery in Databases (KDD):

    • KDD is a non-trivial process of identifying valid, novel, potentially useful, and ultimately understandable patterns in data (Fayyad et al., 1996).

    • It involves extracting implicit, previously unknown, and likely valuable information from data, typically devising models and patterns from large databases (Sarker et al., 2000; Bramer, 2007).

    • Focuses on revealing hidden insights from data through model and pattern creation from large datasets.

  • Data Mining (DM):

    • Merges conventional data analysis methods with algorithms that identify patterns in vast data amounts while adhering to computational efficiency constraints (Tan et al., 2006).

    • Utilizes algorithms to efficiently identify patterns, often merging traditional data analysis methods.

    • Although often used interchangeably with KDD, DM is a more focused process within the broader KDD framework.

  • Machine Learning (ML):

    • Enables computers to learn from data without explicit programming (Mitchell, 1997).

    • Uncovers knowledge that would be challenging or impossible to extract through traditional statistical analysis or manual inspection.

    • ML is the process where computer systems utilize computer science and statistical techniques to progressively improve their performance on analytic tasks (Samuel, 1959).

    • While often considered a subfield of DM, ML emphasizes automation, which may not always apply in traditional data mining.

Interrelationship of DM and ML

  • The literature often uses data mining and machine learning interchangeably; classical statistics serve as the foundation for both.

  • Both domains share a symbiotic evolution, influencing and borrowing methodologies from each other (Shu and Ye, 2023).

  • ML's introduction in the late 1950s marked a transformative shift, enabling algorithms to learn and refine based on data inputs (Shu and Ye, 2023).

Areas of Machine Learning

  • Machine Learning is categorized into two major areas:

    • Supervised Learning:

    • Involves making predictions about new data related to specific phenomena.

    • Unsupervised Learning:

    • Reveals relationships and patterns within data that are not immediately obvious.

Knowledge Discovery Process (KDD Process)

  • The KDD process framework, as proposed by Usama Fayyad, Gregory Piatetsky-Shapiro, and Padhraic Smyth in 1996, is structured and iterative:

    1. Data Collection:

    • Gathers data from diverse sources and formats: structured databases, unstructured text documents, multimedia files, and sensor data.

    • Data extraction can occur manually or automatically through various techniques.

    1. Data Integration and Selection:

    • Due to the heterogeneous nature of data sources, an initial integration and selection stage is necessary to consolidate data into a designated repository.

    1. Data Preprocessing:

    • This stage enhances data suitability for analysis via noise removal, dealing with missing data, and defining strategies for incomplete information.

    1. Pattern Discovery:

    • Involves uncovering latent patterns and relationships using machine learning algorithms and data mining techniques.

    • Models or rules represent hidden structures and relationships within the data.

    1. Interpretation and Evaluation:

    • This assesses the validity and significance of the extracted knowledge, employing evaluation techniques to ensure alignment with research objectives.

    1. Iteration:

    • If necessary, revisiting prior stages for further data preparation, algorithm selection, or refinement ensures reliability and generalizability in the findings.

    1. Consolidation of Knowledge:

    • The process culminates in integrating knowledge into other systems, visualizing patterns, or documenting them for future use.

CRISP-DM Process

  • CRISP-DM: The CRoss-Industry Standard Process for Data Mining, is another structured methodology defined by Pete Chapman in 1999, consisting of several key phases:

  1. Business Understanding:

    • Focuses on bridging business needs with data-driven solutions.

    • Involves understanding project requirements from a business perspective and translating them into a data mining problem.

    • Use of tools like Decision Model and Notation (DMN) standard ensures alignment between business goals and data mining activities.

  2. Data Understanding:

    • Initial data collection leads to exploring the data’s structure, assessing quality, and uncovering insights—this phase mitigates risks and aids informed business decisions.

    • It is critical to understand data characteristics for valid inferences and robust modeling.

  3. Data Preparation:

    • Involves transforming raw data into a refined dataset specific for analysis, including:

      • Consolidating data sources

      • Standardizing names for consistency

      • Cleaning data of errors/inconsistencies

      • Handling outliers

      • Transforming data for analysis tools

      • Partitioning data into training, testing, and validation sets.

    • Often automated within Extract, Transform, Load (ETL) processes for efficiency.

  4. Modeling:

    • Selection, application, and optimization of various modeling techniques suitable for the prepared dataset.

    • Involves experimenting with multiple algorithms and fine-tuning them to determine the optimal approach for the data mining problem.

  5. Evaluation:

    • Assessing the quality and validity of chosen model(s), analyzing predictive accuracy and model stability.

    • Evaluation not only involves technical metrics but also aligns model outcomes with business objectives.

  6. Deployment:

    • Translating model insights into actionable outcomes marks the transition from data analysis to practical application.

    • May involve generating reports or integrating models into operational systems. Close collaboration is vital for effective model communication and support.

  7. Feedback and Iteration:

    • A continuous feedback loop allows data from deployed models to refine and improve future iterations, emphasizing that data mining is an iterative journey rather than a linear process.

    • A balance must be struck between perfection and timely deployment, ensuring that models are deployed at their best potential, understanding that incremental gains must outweigh costs.