Comprehensive Study Notes on Data Mining, Decision Trees, Association Rules, Clustering, and Data Visualization

Positioning Data Mining within Business Process Management and Process Mining

• Interrelationship of Disciplines:
  • Business Process Management (BPM): Focuses on managing, executing, and improving operational business processes through governance, model-driven techniques, and structured workflows.
  • Process Mining: Acts as the bridge connecting data-driven science (data mining) with process-centric disciplines (BPM). It leverages historical event logs from operational enterprise systems to discover, monitor, and enhance actual processes.
    • Process Discovery: Automatically produces process models (e.g., Petri nets, BPMN diagrams) from raw event logs without subjective human bias.
    • Conformance Checking: Compares an a priori normative process model with real-world execution logs to detect discrepancies, bottlenecks, deviations, and regulatory compliance issues.
    • Predictive Analytics: Forecasts remaining process cycle times, probabilities of undesirable outcomes (e.g., loan default, customer cancellation), and optimal path routing for active running cases.
  • Data Mining: Encompasses algorithms designed to extract non-trivial, implicit, previously unknown, and potentially useful patterns, structures, and associations from large tabular datasets. Core subdisciplines include:
    • Supervised Learning: Decision tree learning, classification, and regression.
    • Unsupervised Learning: Association rule learning, sequence mining, episode mining, and cluster analysis.

Supervised Learning and Decision Tree Induction

• Conceptual Foundations of Classification:

  • Supervised Learning Paradigm: Learning algorithms operate on datasets where every training record possesses a known target outcome (label). The objective is to establish an inductive mapping function from feature variables to the target label.
  • Variable Taxonomy:
    • Predictor Variables (Independent Variables / Features / Attributes): Attributes used to explain or forecast an outcome (e.g., age, income, medical habits, examination scores).
    • Response Variable (Dependent Variable / Label / Target): The primary outcome being predicted (e.g., loan default status, academic graduation distinction, disease presence).
    • Irrelevant or Noisy Variables: Columns containing random IDs or spurious features lacking theoretical, physical, or logical linkage to the response variable (e.g., Record_ID, Favorite_Color, Shoe_Size, Zodiac_Sign). Including these introduces noise and increases the risk of overfitting.
    • Context-Dependent Role Shifting: Variables shift roles based on the business objective. In an employee attrition prediction model, Attrition (Yes/No) is the label, while Coffee_Cups/Day serves as a predictor of stress or workload. If the objective pivots to predicting daily coffee consumption, Coffee_Cups/Day becomes the continuous response variable, while Attrition becomes an explanatory feature indicating disengagement or distress.

• Decision Tree Architecture and Classification Mechanics:

  • A decision tree is an upside-down tree structure consisting of:
    • Root Node: The top-level parent node containing all training instances prior to any partition.
    • Internal (Decision) Nodes: Intermediate evaluation points where an attribute condition is tested (e.g., $\text{Credit\_Score} < 600$, $\text{smoker} = \text{yes}$, $\text{weight} \ge 90\,\text{kg}$).
    • Branches: Edges representing the outcome of a decision node test, directing records along mutually exclusive pathways.
    • Leaf (Terminal) Nodes: End nodes assigning a predicted class label, denoted alongside instance counts and misclassification tallies, such as $\text{young } (195/11)$, which denotes that of $195$ instances reaching that leaf, $11$ are misclassified errors.
  • Instance Traversal Procedure: An unseen instance enters at the root node, iteratively evaluates its corresponding attribute values against split predicates at internal nodes, and terminates at a leaf node that assigns the predicted class label.

• Exemplar Datasets and Classification Logic:

  • Dataset 1: Lifestyle Effect on Life Expectancy:
    • Predictors: drinker (yes/no), smoker (yes/no), weight (continuous kg).
    • Response Variable: Binary discretized age class: $\ge 70 = \text{"old"}$, $< 70 = \text{"young"}$.
    • Learned Tree Rules:
      1. If $\text{smoker} = \text{yes}$, classify as young (195/11) ($184$ true young, $11$ misclassified old).
      2. If $\text{smoker} = \text{no}$ and $\text{drinker} = \text{no}$, classify as old (65/2) ($63$ true old, $2$ misclassified young).
      3. If $\text{smoker} = \text{no}$ and $\text{drinker} = \text{yes}$ and $\text{weight} < 90\,\text{kg}$, classify as old (219/34) ($185$ true old, $34$ misclassified young).
      4. If $\text{smoker} = \text{no}$ and $\text{drinker} = \text{yes}$ and $\text{weight} \ge 90\,\text{kg}$, classify as young (381/55) ($326$ true young, $55$ misclassified old).

*   *Case Classifications:*
    *   *Mary Jones:* Predictors: `drinker = yes`, `smoker = no`, `weight = 70 kg`, `age = 85 year`. Follows branch: $\text{smoker}=\text{no} \rightarrow \text{drinker}=\text{yes} \rightarrow \text{weight} < 90 \rightarrow \text{Leaf: "old"}$. Actual age is $85$ ($\ge 70$). Result: **Correctly Classified**.
    *   *Sue Smith:* Predictors: `drinker = no`, `smoker = no`, `weight = 60 kg`, `age = 35 year`. Follows branch: $\text{smoker}=\text{no} \rightarrow \text{drinker}=\text{no} \rightarrow \text{Leaf: "old"}$. Actual age is $35$ ($< 70$). Result: **Misclassified** (actual young classified as old).

• Dataset 2: Course Performance on Academic Graduation:
  • Predictors: Exam marks for linear algebra, logic, programming, operations research, workflow systems, and study duration (months).
  • Response Variable: Graduation award: failed, passed, or cum laude.
  • Learned Tree Rules:
    • If $\text{logic} = \text{"-"}$ (missing/unattempted), classify as failed (79/10).
    • If $\text{logic} < 8$ and $\text{linear algebra} \ge 6$, classify as passed (87/11).
    • If $\text{logic} < 8$ and $\text{linear algebra} < 6$ and $\text{operations research} \ge 6$, classify as passed (82/7).
    • If $\text{logic} < 8$ and $\text{linear algebra} < 6$ and $\text{operations research} < 6$, classify as failed (101/8).
    • If $\text{logic} \ge 8$ and $\text{programming} \ge 7$, classify as cum laude (20/2).
    • If $\text{logic} \ge 8$ and $\text{programming} < 7$ and $\text{linear algebra} \ge 6$, classify as passed (31/7).
    • If $\text{logic} \ge 8$ and $\text{programming} < 7$ and $\text{linear algebra} < 6$, classify as failed (20/4).

• Dataset 3: Point-of-Sale Beverage Purchases and Pastry Complements:
  • Predictors: Quantities of beverages ordered: cappuccino, latte, espresso, americano, ristretto, tea, bagel.
  • Response Variable: Binary consumption: muffin ($\ge 1$ items) versus no muffin ($0$ items).
  • Learned Tree Rules:
    • If $\text{tea} \ge 1$, classify as muffin (30/1).
    • If $\text{tea} = 0$ and $\text{latte} = 0$, classify as no muffin (189/10).
    • If $\text{tea} = 0$ and $\text{latte} \ge 2$, classify as muffin (4/0).
    • If $\text{tea} = 0$ and $\text{latte} = 1$ and $\text{espresso} = 0$, classify as muffin (6/2).
    • If $\text{tea} = 0$ and $\text{latte} = 1$ and $\text{espresso} \ge 1$, classify as no muffin (11/3).
  • Receipt Validation Checks:
    • Check #689561 (Customer 1): cappuccino = 2, latte = 3, muffin = 1, bagel = 2. Model evaluates: $\text{tea} = 0 \rightarrow \text{latte} = 3\,(\ge 2) \rightarrow \text{muffin}$. The actual receipt contains $1$ muffin. Result: Correctly Classified.
    • Check #689561 (Customer 2): bagel = 1, latte = 1, ristretto = 1. Receipt has $0$ tea, $1$ latte, $0$ espresso. Model evaluates: $\text{tea} = 0 \rightarrow \text{latte} = 1 \rightarrow \text{espresso} = 0 \rightarrow \text{muffin}$. The actual check contains no muffin. Result: Misclassified (False Positive).
    • Unknown Receipt: bagel = 1, latte = 2, cappuccino = 1. Evaluation: $\text{tea} = 0 \rightarrow \text{latte} = 2\,(\ge 2) \rightarrow \text{muffin}$. Prediction: Customer consumed a muffin.

• Compatibility Check of Simplified Decision Tree on Credit Loan Defaults:
  • Records Under Evaluation:
    • Case 001: Age = 25, Income_Level = Medium, Credit_Score = 650, Owns_House = No, Loan_Default = No.
    • Case 002: Age = 40, Income_Level = High, Credit_Score = 720, Owns_House = Yes, Loan_Default = No.
    • Case 003: Age = 30, Income_Level = Low, Credit_Score = 580, Owns_House = No, Loan_Default = Yes.
    • Case 004: Age = 35, Income_Level = Medium, Credit_Score = 690, Owns_House = Yes, Loan_Default = No.
    • Case 005: Age = 50, Income_Level = High, Credit_Score = 750, Owns_House = Yes, Loan_Default = No.
  • Tree Logic: If $\text{Credit\_Score} < 600 \rightarrow \text{Yes}$. Else if $\text{Income\_Level} = \text{Low} \rightarrow \text{Yes}$, else $\text{No}$.
  • Execution Walkthrough:
    • Case 001: $\text{Credit\_Score} = 650\,(\ge 600) \rightarrow \text{Income} = \text{Medium} \rightarrow \text{Prediction: No}$. (Matches ground truth: No).
    • Case 002: $\text{Credit\_Score} = 720\,(\ge 600) \rightarrow \text{Income} = \text{High} \rightarrow \text{Prediction: No}$. (Matches ground truth: No).
    • Case 003: $\text{Credit\_Score} = 580\,(< 600) \rightarrow \text{Prediction: Yes}$. (Matches ground truth: Yes).
    • Case 004: $\text{Credit\_Score} = 690\,(\ge 600) \rightarrow \text{Income} = \text{Medium} \rightarrow \text{Prediction: No}$. (Matches ground truth: No).
    • Case 005: $\text{Credit\_Score} = 750\,(\ge 600) \rightarrow \text{Income} = \text{High} \rightarrow \text{Prediction: No}$. (Matches ground truth: No).
  • Compatibility Decision: Yes, the decision tree model is fully compatible with the dataset ($100\%$ accuracy across all sample records). Unused columns (Age, Owns_House) are redundant features.

Information Theory and the Mathematical Mechanics of Tree Construction

• Mathematical Concept of Entropy:
  • Entropy ($E$) quantifies the degree of uncertainty, disorder, or impurity within a set of instances.
  • It is mathematically inversely proportional to data "compressibility" or "zippability". Homogeneous, pure subsets exhibit minimal entropy (high compressibility), while evenly mixed distributions exhibit maximal entropy (zero compressibility).
  • Mathematical Logarithmic Axioms:

        $\log_2(x) = y \iff 2^y = x$

        $\log_2(2^n) = n$

        $\log_2\left(\frac{1}{2^n}\right) = -n$

        $\log_2(1) = 0$

        $\log_2(2) = 1$

        $\log_2(8) = 3$

        $\log_2(1024) = 10$

        $\log_2(0.125) = \log_2\left(\frac{1}{8}\right) = -3$

        $\log_2(0.75) \approx -0.415$

• Shannon Entropy Formula:
  • Let a dataset or node partition contain $n$ instances across $k$ discrete class values, where each class $i$ has count $c_i \ge 1$, such that $n = \sum_{i=1}^k c_i$. The proportion belonging to class $i$ is $p_i = \frac{c_i}{n}$.

        $E = -\sum_{i=1}^k p_i \log_2(p_i)$

*   *Worked Prototypical Scenarios:*
    *   *Balanced Binary Split ($3$ red, $3$ green; $n=6$):*

            $E = -\left(\frac{3}{6}\log_2\left(\frac{3}{6}\right) + \frac{3}{6}\log_2\left(\frac{3}{6}\right)\right) = -\left(\frac{1}{2}(-1) + \frac{1}{2}(-1)\right) = 1.0$

    *   *Pure Binary Subset ($2$ red, $0$ green; $n=2$):*

            $E = -\left(\frac{2}{2}\log_2\left(\frac{2}{2}\right)\right) = -(1 \times 0) = 0.0$

    *   *Skewed Binary Distribution ($1$ red, $3$ green; $n=4$):*

            $E = -\left(\frac{1}{4}\log_2\left(\frac{1}{4}\right) + \frac{3}{4}\log_2\left(\frac{3}{4}\right)\right) = -\left(\frac{1}{4}(-2) + \frac{3}{4}(-0.415)\right) = -(-0.5 - 0.31125) \approx 0.811$

• Information Gain and Multi-Level Tree Optimization:
  • The recursive goal of greedy decision tree algorithms (e.g., ID3, C4.5) is to select the split attribute that maximizes Information Gain ($IG$), defined as the reduction in entropy from the parent node to the weighted average entropy of the generated child nodes:

        $IG(D, A) = E(D) - \sum_{v \in \text{Values}(A)} \frac{|D_v|}{|D|} E(D_v)$

*   *Step-by-Step Derivation on Smoking-Lifestyle Dataset ($N=860$ total instances):*
    *   *Root Node Distribution:* $546$ young, $314$ old ($N=860$).

            $E(\text{Root}) = -\frac{546}{860}\log_2\left(\frac{546}{860}\right) - \frac{314}{860}\log_2\left(\frac{314}{860}\right) = 0.946848$

    *   *Split on Attribute `smoker`:*
        *   Branch `smoker = yes`: $195$ instances ($184$ young, $11$ old).

                $E(\text{smoker}=\text{yes}) = -\frac{184}{195}\log_2\left(\frac{184}{195}\right) - \frac{11}{195}\log_2\left(\frac{11}{195}\right) = 0.313027$

        *   Branch `smoker = no`: $665$ instances ($362$ young, $303$ old).

                $E(\text{smoker}=\text{no}) = -\frac{362}{665}\log_2\left(\frac{362}{665}\right) - \frac{303}{665}\log_2\left(\frac{303}{665}\right) = 0.994314$

        *   *Weighted Average Entropy of Split:*

                $E(\text{Layer 2}) = \frac{195}{860}(0.313027) + \frac{665}{860}(0.994314) = 0.839836$

        *   *Information Gain:*

                $IG(\text{smoker}) = 0.946848 - 0.839836 = 0.107012$

                Note: Although the majority classification of both subsets remains young, significant information gain occurs because uncertainty within the smoker cohort drops dramatically. * Further Partition on Attribute drinker (Sub-splitting smoker = no, $N=665$): * Branch drinker = yes: $600$ instances ($360$ young, $240$ old).

                $E = -\frac{360}{600}\log_2\left(\frac{360}{600}\right) - \frac{240}{600}\log_2\left(\frac{240}{600}\right) = 0.970951$

        *   Branch `drinker = no`: $65$ instances ($2$ young, $63$ old).

                $E = -\frac{2}{65}\log_2\left(\frac{2}{65}\right) - \frac{63}{65}\log_2\left(\frac{63}{65}\right) = 0.198234$

        *   *Weighted Average Entropy across the three terminal cohorts:*

                $E = \frac{195}{860}(0.313027) + \frac{600}{860}(0.970951) + \frac{65}{860}(0.198234) = 0.763368$

        *   *Information Gain of Second Split:*

                $IG(\text{drinker}) = 0.839836 - 0.763368 = 0.076468$

*   *Pathological Scenario: Zero Information Gain Split:*
    *   *Layer 1 (Root):* Total $N=160$, with $100$ pass, $60$ fail.

            $E(\text{Layer 1}) = -\left(\frac{100}{160}\log_2\left(\frac{100}{160}\right) + \frac{60}{160}\log_2\left(\frac{60}{160}\right)\right) = -(0.625\log_2(0.625) + 0.375\log_2(0.375)) = 0.954$

    *   *Split on Attribute `smoke`:*
        *   `smoke = yes`: Total $N=40$, with $25$ pass, $15$ fail.
            *   Proportions: $\frac{25}{40} = 0.625$, $\frac{15}{40} = 0.375$.
            *   Entropy: $E = 0.954$.
        *   `smoke = no`: Total $N=120$, with $75$ pass, $45$ fail.
            *   Proportions: $\frac{75}{120} = 0.625$, $\frac{45}{120} = 0.375$.
            *   Entropy: $E = 0.954$.
    *   *Layer 2 Weighted Average Entropy:*

            $E(\text{Layer 2}) = \left(\frac{40}{160}\right)(0.954) + \left(\frac{120}{160}\right)(0.954) = 0.954$

    *   *Information Gain:*

            $IG = E(\text{Layer 1}) - E(\text{Layer 2}) = 0.9544 - 0.9544 = 0$

    *   *Evaluation:* It was **not worthwhile** to partition the dataset on `smoke`. The class distribution ($62.5\%$ pass, $37.5\%$ fail) remained perfectly identical across both branches, meaning the split resolved zero uncertainty.

• Decision Tree Inductive Algorithm Routine:

  1. Initialize the tree with a root node containing all instances.
  2. Evaluate all candidate predictor attributes across every active un-split node.
  3. Calculate the Information Gain generated by splitting on each attribute.
  4. Select the candidate attribute yielding the maximal Information Gain above an established minimum threshold.
  5. Partition instances into subsets matching the selected split predicate and generate child nodes.
  6. Recursively repeat steps 2–5 on child nodes until stopping criteria are satisfied.
  7. Return the finalized tree structure.

• Hyperparameters, Algorithmic Variations, and Pruning:

  • Pre-Pruning Stopping Constraints:
    • Minimum leaf size / node size before or after splitting.
    • Information gain minimum threshold (splits below threshold are terminated).
    • Maximal tree depth limits.
    • Restricting identical attribute labels from reappearing along a decision path.
  • Alternative Splitting Criteria:
    • Gini Index of Diversity:

            $G = \sum_{i=1}^k p_i(1 - p_i) = 1 - \sum_{i=1}^k (p_i)^2$

*   *Continuous Variable Handling:* Finding optimal numeric thresholds (e.g., $\text{weight} < 90\,\text{kg}$ vs. $\ge 90\,\text{kg}$) by sorting unique values and calculating candidate cut-point gains.
*   *Post-Pruning:* Pruning backward from leaves toward the root by removing subtrees that fail to improve generalized statistical power on validation data, mitigating overfitting.

• Application in Process Mining (Decision Point Analysis):
  • Discovered business process models contain routing choice nodes (decision points) where alternate execution paths diverge.
  • By replaying recorded historical event logs over a discovered Petri net or BPMN model, state data attributes associated with each process instance at the decision point become predictor variables.
  • A decision tree is trained at each choice junction to discover operational rules governing routing, answering critical questions such as:
    • What data attributes determine whether a travel request requires senior management support, additional motivation, or immediate approval?
    • Which case features correlate with request rejection or cycle time budget overruns?

Model Evaluation, Performance Metrics, and Validation Protocols

• The Binary Confusion Matrix:

  • Evaluates classification models by tabulating cross-classifications between predicted outcomes and ground-truth values:
    • True Positives ($TP$): Positive instances correctly predicted as positive.
    • True Negatives ($TN$): Negative instances correctly predicted as negative.
    • False Positives ($FP$): Negative instances incorrectly predicted as positive (Type I error).
    • False Negatives ($FN$): Positive instances incorrectly predicted as negative (Type II error).
  • Marginal Totals:
    • Total Actual Positives: $P = TP + FN$
    • Total Actual Negatives: $N = FP + TN$
    • Total Predicted Positives: $P' = TP + FP$
    • Total Predicted Negatives: $N' = FN + TN$
    • Total Instances: $K = TP + TN + FP + FN$

• Mathematical Performance Metrics:

  • Error Rate:

        $\text{Error Rate} = \frac{FP + FN}{K}$

*   **Accuracy:**

        $\text{Accuracy} = \frac{TP + TN}{K}$

*   **Precision:** Proportion of positive identifications that are truly correct:

        $\text{Precision} = \frac{TP}{P'} = \frac{TP}{TP + FP}$

*   **Recall (Sensitivity / True Positive Rate):** Proportion of actual positives captured by the model:

        $\text{Recall} = \frac{TP}{P} = \frac{TP}{TP + FN}$

*   **F1-Score:** The harmonic mean of precision and recall, balancing precision and recall under class imbalance:

        $\text{F1-Score} = \frac{2 \times \text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}$

• Comparative Performance Derivations:

  • Lifestyle Dataset Evaluation (Positive = Young, Negative = Old):
    • Prior to Splitting (Root Baseline): All $860$ instances classified as positive (young).
      • $TP = 546$, $TN = 0$, $FP = 314$, $FN = 0$.
      • $\text{Precision} = \frac{546}{546 + 314} = 0.635$
      • $\text{Recall} = \frac{546}{546 + 0} = 1.000$
      • $\text{F1-Score} = \frac{2 \times 0.635 \times 1.000}{0.635 + 1.000} = 0.777$
    • After Multi-Attribute Tree Splitting: Partitioning creates a dedicated negative leaf (old) for non-smoking, non-drinking instances.
      • $TP = 544$, $TN = 63$, $FP = 251$, $FN = 2$.
      • $\text{Precision} = \frac{544}{544 + 251} = \frac{544}{795} \approx 0.684$
      • $\text{Recall} = \frac{544}{544 + 2} = \frac{544}{546} \approx 0.996$
      • $\text{F1-Score} = \frac{2 \times 0.684 \times 0.996}{0.684 + 0.996} \approx 0.811$
      • Conclusion: Splitting successfully increases Precision ($0.635 \rightarrow 0.684$) and overall F1-Score ($0.777 \rightarrow 0.811$) while identifying negative cases.
  • Pathological Zero-Information-Gain Model Evaluation (Layer 2):
    • Because both branches predict pass, predicted fails are zero ($TN = 0$, $FN = 0$).
    • Actual Pass = $100$, Actual Fail = $60$, $K = 160$.
    • $TP = 100$, $TN = 0$, $FP = 60$, $FN = 0$.
    • $\text{Error Rate} = \frac{60 + 0}{160} = 0.375$
    • $\text{Accuracy} = \frac{100 + 0}{160} = 0.625$
    • $\text{Precision} = \frac{100}{100 + 60} = 0.625$
    • $\text{Recall} = \frac{100}{100 + 0} = 1.0$
    • $\text{F1-Score} = \frac{2 \times 0.625 \times 1.0}{0.625 + 1.0} = \frac{1.25}{1.625} \approx 0.769$

• Overfitting, Underfitting, and Validation Methodologies:

  • Overfitting: Occurs when a model incorporates noise, idiosyncrasies, and specific random artifacts from training data rather than underlying statistical truths.
    • Conceptual Illustration: Memorizing ten friends' heights using birth date and eye color yields $100\%$ training accuracy (e.g., "If birth date = 16-05-1998 and eye color = blue, height = $172.8\,\text{cm}$"), but collapses entirely on unseen test cases.
  • Underfitting: Occurs when a model is excessively crude or constrained, failing to capture meaningful structural patterns (e.g., "If gender = male, height $> 1\,\text{meter}$").
  • Split Validation: The source dataset is partitioned into an independent training set (used for tree induction) and a test set (used exclusively to evaluate performance metrics).
  • $k$-Fold Cross-Validation:
    1. The complete dataset is randomly partitioned into $k$ equally sized, mutually exclusive subsamples (folds).
    2. In each of $k$ iterations, a single fold is retained as the testing model evaluation set, while the remaining $k-1$ folds are aggregated to train the model.
    3. The performance results of all $k$ iterations are averaged to yield an overall metric less susceptible to sampling bias.
  • Operational Complications:
    • Concept Drift: Relationships between features and response labels change over time due to external shifts (e.g., macroeconomic fluctuations, seasonal shifts), causing static models to decay.
    • Missing Negative Classes (One-Class Data): Datasets that document positive events but lack negative records (e.g., logs tracking sick patients who complain, while satisfied or non-sick patients go unrecorded).

Unsupervised Learning: Association Rule Mining and Pattern Extraction

• Core Concepts and Market Basket Analysis:
  • Unsupervised Nature: Unsupervised algorithms extract latent structural affinities without a designated target/response label.
  • Market Basket Analysis: Analyzes co-occurrence patterns in transactional point-of-sale systems to determine which items customers purchase together.
  • Urban Legend of Retail Analytics: The widely cited retail anecdote where supermarket analysis supposedly discovered that young fathers buying diapers on Friday afternoons also bought beer, prompting the co-location of beer and diaper aisles to increase sales.
  • Rule Structure: Expressed as directional implications:

        $X \Rightarrow Y$

        Where $X$ denotes the antecedent (premise itemset) and $Y$ denotes the consequent (conclusion itemset), with $X \cap Y = \emptyset$.

• Mathematical Formulations of Rule Quality Metrics:
  • Let $N$ be the total number of transactions, $N_X$ be transactions containing itemset $X$, $N_Y$ be transactions containing $Y$, and $N_{X \cap Y}$ (or $N_{X \cup Y}$) be transactions containing both $X$ and $Y$.
  • Support: Measures how frequently the complete rule appears in the entire dataset ($0 \le \text{Support} \le 1$):

        $\text{support}(X \Rightarrow Y) = \frac{N_{X \wedge Y}}{N} = \frac{N_{X \cup Y}}{N}$

*   **Confidence:** Measures how frequently items in $Y$ appear in transactions known to contain $X$ ($0 \le \text{Confidence} \le 1$):

        $\text{confidence}(X \Rightarrow Y) = \frac{N_{X \wedge Y}}{N_X} = \frac{N_{X \cup Y}}{N_X}$

*   **Lift:** Quantifies the directional dependency between $X$ and $Y$ relative to their baseline independence:

        $\text{lift}(X \Rightarrow Y) = \frac{N_{X \wedge Y} / N}{(N_X / N)(N_Y / N)} = \frac{N_{X \wedge Y} \cdot N}{N_X \cdot N_Y}$

*   *Interpretation of Lift Values:*
    *   $\text{lift}(X \Rightarrow Y) > 1$: $X$ and $Y$ are **positively correlated** (the presence of $X$ increases the probability of $Y$).
    *   $\text{lift}(X \Rightarrow Y) \approx 1$: $X$ and $Y$ are **statistically independent** (no affinity).
    *   $\text{lift}(X \Rightarrow Y) < 1$: $X$ and $Y$ are **negatively correlated** (substitutes; purchasing $X$ actively decreases the probability of $Y$).

• Worked Numerical Validations of Association Rules:

  • Retail Diaper and Beer Transaction Scenario ($N=100$ customers):

    • $9$ customers purchase exclusively Hoegaarden.
    • $40$ customers purchase exclusively Pampers.
    • $50$ customers purchase Pampers and Dommelsch.
    • $1$ customer purchases Pampers, Hoegaarden, and Dommelsch.
    • Item Marginal Frequencies:
      • $N_{\text{Pampers}} = 40 + 50 + 1 = 91$
      • $N_{\text{Dommelsch}} = 50 + 1 = 51$
      • $N_{\text{Hoegaarden}} = 9 + 1 = 10$
      • $N_{\text{Pampers} \cap \text{Dommelsch}} = 50 + 1 = 51$
      • $N_{\text{Pampers} \cap \text{Hoegaarden}} = 1$
      • $N_{\text{Pampers} \cap \text{Dommelsch} \cap \text{Hoegaarden}} = 1$
    • Metric Calculations for Candidate Rules:
      1. Rule: $\text{\{Pampers\}} \Rightarrow \text{\{Dommelsch\}}$
         • $\text{Support} = \frac{51}{100} = 0.51$
         • $\text{Confidence} = \frac{51}{91} \approx 0.56$
         • $\text{Lift} = \frac{51 \times 100}{91 \times 51} = \frac{100}{91} \approx 1.10$
      2. Rule: $\text{\{Pampers\}} \Rightarrow \text{\{Hoegaarden\}}$
         • $\text{Support} = \frac{1}{100} = 0.01$
         • $\text{Confidence} = \frac{1}{91} \approx 0.01$
         • $\text{Lift} = \frac{1 \times 100}{91 \times 10} = \frac{100}{910} \approx 0.11$
      3. Rule: $\text{\{Dommelsch\}} \Rightarrow \text{\{Pampers\}}$
         • $\text{Support} = \frac{51}{100} = 0.51$
         • $\text{Confidence} = \frac{51}{51} = 1.00$
         • $\text{Lift} = \frac{51 \times 100}{51 \times 91} = \frac{100}{91} \approx 1.10$
      4. Rule: $\text{\{Pampers, Dommelsch\}} \Rightarrow \text{\{Hoegaarden\}}$
         • $\text{Support} = \frac{1}{100} = 0.01$
         • $\text{Confidence} = \frac{1}{51} \approx 0.02$
         • $\text{Lift} = \frac{1 \times 100}{51 \times 10} = \frac{100}{510} \approx 0.20$

  • Employee Workplace Initiatives Survey ($N=200$ employees):

    *   *Rule to Analyze:* $\text{\{Time Management Workshop, Leadership Seminar\}} \Rightarrow \text{\{Team-Building Activity\}}$
    *   *Aggregations:*
        *   Total attending Time Management & Leadership: $N_{\{TM, LS\}} = 35 + 5 = 40$.
        *   Total attending Team-Building: $N_{\{TB\}} = 30 + 25 + 15 + 5 = 75$.
        *   Total attending all three: $N_{\{TM, LS, TB\}} = 5$.
    *   *Metrics & Organizational Interpretations:*
        *   $\text{Support} = \frac{5}{200} = 0.025\,(2.5\%)$: Represents the proportion of employees participating in all three initiatives, indicating that enterprise-wide multi-program engagement is rare.
        *   $\text{Confidence} = \frac{5}{40} = 0.125\,(12.5\%)$: Demonstrates that among employees attending both Time Management and Leadership, only $12.5\%$ choose to participate in Team-Building.
        *   $\text{Lift} = \frac{5 \times 200}{40 \times 75} = \frac{1000}{3000} \approx 0.333$: Because $\text{Lift} < 1$, a substantial negative association exists. Employees participating in professional development seminars avoid social team-building activities compared to baseline employee participation ($\frac{75}{200} = 37.5\%$).

• Algorithmic Pruning and the Apriori Principle:
  • Brute-Force Bottleneck: In a domain with $d$ unique items, $2^d$ possible itemsets exist. Evaluating every combination causes combinatorial explosion in computation and produces too many uninterpretable rules.
  • Anti-Monotonicity (Apriori Property): Any subset of a frequent itemset must also be frequent:

        $Y \subseteq X \implies \frac{N_Y}{N} \ge \frac{N_X}{N}$

*   *Algorithmic Pruning Logic:* If an itemset $Y$ fails to meet the minimum support threshold ($minsup$), it is discarded alongside all of its supersets ($X \supset Y$), pruning large portions of the search space.
*   *Apriori Execution Pipeline:*
    1. Transform transactional database into binary indicator format (binonimal flags: true/false).
    2. Generate all frequent itemsets $Z$ such that $\frac{N_Z}{N} \ge minsup$ and $|Z| \ge 2$.
    3. For every frequent itemset $Z$, evaluate all non-empty partitions into antecedent $X$ and consequent $Y$ ($X \cup Y = Z, X \cap Y = \emptyset$).
    4. Retain rules satisfying $\text{confidence}(X \Rightarrow Y) \ge minconf$, while discarding others.

• Restaurant Transaction Case Study (5000 Dining Parties):
  • Evaluating orders across $12$ Italian food and drink items.
  • Setting baseline thresholds to $minsup = 0.10$ and $minconf = 0.50$ yields $153$ frequent itemsets and $743$ association rules.
  • Tightening constraints to $minsup = 0.30$ and $minconf = 0.90$ filters the output to $61$ high-confidence rules:
    • $R_1 = \text{\{espresso, pizza siciliana, pizza romana\}} \Rightarrow \text{\{vino bianco\}}$
      • $\text{Support} = 0.326$, $\text{Confidence} = 0.904$, $\text{Lift} = 1.752$
    • $R_2 = \text{\{vino rosso, vino bianco, pizza romana\}} \Rightarrow \text{\{espresso, pizza siciliana\}}$
      • $\text{Support} = 0.303$, $\text{Confidence} = 0.904$, $\text{Lift} = 2.291$
    • $R_3 = \text{\{vino rosso, vino bianco\}} \Rightarrow \text{\{pizza siciliana\}}$
      • $\text{Support} = 0.354$, $\text{Confidence} = 0.904$, $\text{Lift} = 1.886$

• Strategic Scenario Matching (TechHaven E-Commerce Case Study):

*   *Rule A: $\text{\{AA Batteries\}} \Rightarrow \text{\{AAA Batteries\}}$ ($\text{Support}=45.0\%$, $\text{Confidence}=52.0\%$, $\text{Lift}=1.05$):*
    *   *Scenario:* Mass-Market Store Redesign for the homepage "frequently bought together" widget.
    *   *Justification:* Reaches nearly half of all daily website visitors due to its high support ($45.0\%$), driving high-volume add-on sales across a broad audience.
*   *Rule B: $\text{\{Smart 4K TV\}} \Rightarrow \text{\{Surround Soundbar\}}$ ($\text{Support}=12.0\%$, $\text{Confidence}=65.0\%$, $\text{Lift}=4.20$):*
    *   *Scenario:* High-Profit Holiday Bundle.
    *   *Justification:* Connects two high-ticket items with moderate support ($12.0\%$) and strong positive correlation ($\text{Lift}=4.20$, $\text{Confidence}=65.0\%$) to drive combined revenue.
*   *Rule C: $\text{\{High-End GPU, Motherboard\}} \Rightarrow \text{\{Liquid CPU Cooler\}}$ ($\text{Support}=1.5\%$, $\text{Confidence}=94.0\%$, $\text{Lift}=9.80$):*
    *   *Scenario:* Highly Targeted Email Campaign with limited marketing budget.
    *   *Justification:* Targets a specialized niche ($\text{Support}=1.5\%$) whose cart history makes purchase near-certain ($\text{Confidence}=94.0\%$, $\text{Lift}=9.80$).
*   *Rule D: $\text{\{Wired Gaming Headset\}} \Rightarrow \text{\{Wireless Gaming Headset\}}$ ($\text{Support}=4.0\%$, $\text{Confidence}=8.0\%$, $\text{Lift}=0.35$):*
    *   *Scenario:* Avoiding Product Cannibalization and optimizing warehouse storage.
    *   *Justification:* A lift below 1 ($\text{Lift}=0.35$) and low confidence ($8.0\%$) confirm that these products act as substitutes—purchasing one actively decreases the likelihood of purchasing the other.

• Other Pattern Mining Paradigms:
  • Sequence Mining: Discovers ordered relationships across time (e.g., $\langle \{\text{vino rosso}\}, \{\text{pizza romana, pizza siciliana}\}, \{\text{gelato}\} \rangle \Rightarrow \langle \{\text{espresso}\} \rangle$).
  • Episode Mining: Identifies partially ordered subgraphs of events occurring within execution windows.
  • Limitation: Traditional data mining techniques evaluate flat transactions or sequences and lack the operational context of end-to-end business process models.

Unsupervised Learning: Cluster Analysis

• Foundations of Clustering:

  • Partitions unlabeled instances into groups (clusters) where intra-cluster similarity is maximized and inter-cluster similarity is minimized.
  • Operates without predefined class labels.

• The $k$-Means Algorithm:

  • A partitioned centroid-based clustering algorithm operating via iterative relocation:
    1. Initialization: Select $k$ initial centroids randomly or via regular heuristics within the feature space.
    2. Assignment: Assign each instance to its nearest centroid based on a distance metric (e.g., Euclidean distance):

           $d(\mathbf{x}, \mathbf{c}) = \sqrt{\sum_{j=1}^m (x_j - c_j)^2}$

    3. *Update:* Recompute each centroid's coordinates as the mean vector of all instances assigned to it:

           $\mathbf{c}_k = \frac{1}{|S_k|} \sum_{\mathbf{x} \in S_k} \mathbf{x}$

    4. *Convergence (Fixpoint):* Repeat assignment and update steps until instances no longer change cluster memberships.
*   *Algorithmic Properties:*
    *   Non-deterministic: Random centroid initialization can lead to local optima.
    *   Best-practice protocol requires running multiple trials with randomized seeds and selecting the partition with the lowest within-cluster sum of squares.

• Empirical Restaurant Clustering Case Study ($N=5000$ Dining Parties, $k=2$):
  • Dataset restricted to $6$ core menu items: pizza margherita, pizza siciliana, lasagna, spaghetti carbonara, vino rosso, birra.
  • Cluster Split: Cluster 0 ($2551$ instances) versus Cluster 1 ($2449$ instances).

• Centroid Profiles:
  • Cluster 0 (The Pasta & Beer Segment): lasagna = 0.987, spaghetti carbonara = 0.984, birra = 0.995, pizza margherita = 0.008, pizza siciliana = 0.011, vino rosso = 0.011.
  • Cluster 1 (The Pizza & Red Wine Segment): pizza margherita = 1.026, pizza siciliana = 1.028, vino rosso = 1.046, lasagna = 0.011, spaghetti carbonara = 0.008, birra = 0.007.

*   *Scatter Plot Analysis with Jitter:* Visualizing instances with artificial jitter avoids point overlap and confirms distinct behavioral segregation:
    *   Plotting `pizza margherita` vs `pizza siciliana` isolates Cluster 1 in the upper-right quadrant ($\approx 1$ of each).
    *   Plotting `lasagna` vs `spaghetti carbonara` isolates Cluster 0 in the upper-right quadrant.
    *   Plotting `birra` vs `vino rosso` isolates Cluster 0 on the beer axis and Cluster 1 on the red wine axis.

• Hierarchical Clustering and Process Mining Integration:

  • Agglomerative Hierarchical Clustering: A bottom-up approach where each instance begins as its own cluster. Pairs of clusters are iteratively merged based on linkage distance (e.g., single, complete, average) until all instances form a single root, visualized via a dendrogram.
  • Process Log Partitioning: Applied to decompose complex event logs into homogeneous trace clusters, enabling targeted process discovery models for distinct process variants.

• Multi-Cluster Insight Extraction ($4$-Cluster Behavioral Profiles):

• Cluster Centroid Coordinates:
  • Cluster 0: Spaghetti Carbonara ($1.033$), Lasagna ($0.667$), Beer ($0.233$), Pizza Margherita ($0.033$), Pizza Siciliana ($0$), Red Wine ($0$).
  • Cluster 1: Pizza Siciliana ($0.923$), Pizza Margherita ($0.744$), Red Wine ($0.205$), Lasagna ($0.026$), Spaghetti Carbonara ($0$), Beer ($0$).
  • Cluster 2: Red Wine ($1.500$), Pizza Margherita ($0.438$), Pizza Siciliana ($0.125$), Spaghetti Carbonara ($0.062$), Lasagna ($0$), Beer ($0$).
  • Cluster 3: Beer ($1.267$), Lasagna ($0.333$), Pizza Siciliana ($0$), Pizza Margherita ($0$), Spaghetti Carbonara ($0$), Red Wine ($0$).
• Behavioral Analysis:
  • Intra-Cluster Similarities: Cluster 0 isolates pasta lovers, while Cluster 1 captures pizza lovers.
  • Inter-Cluster Dissimilarities: Cluster 0 orders no pizza, while Cluster 1 orders no pasta. Cluster 2 focuses heavily on red wine ($1.500$) with zero beer, whereas Cluster 3 focuses on beer ($1.267$) with zero wine.
• Menu Engineering Strategies:
  • Targeted Combinations: Introduce a "Pizza & Wine Pairing" for Cluster 2 and a "Beer & Lasagna Combo" for Cluster 3.
  • Menu Layout: Separate the menu into dedicated "Authentic Pizza" and "Traditional Pasta" sections to cater to the distinct preferences of Cluster 0 and Cluster 1.

Comparative Machine Learning Model Selection

• Algorithmic Performance Comparison:
  • Different business contexts require balancing Information Gain, Accuracy, F1-Score, Financial Cost (USD), and Training Time (minutes):

• Deployment Scenarios and Selection Rationale:
  • Scenario 1: Startup under Strict Budget ($< \$100$) Requiring Fast Execution:
    • Selected Model: Decision Tree.
    • Rationale: Training cost is $\$80$ ($< \$100$ budget limit); execution takes $4\text{ min}$; provides the highest accuracy ($88.5\%$) and F1-score ($0.87$) among models priced under $\$100$.
    • Why Others Fail: Deep Learning ($\$1,500$), XGBoost ($\$700$), and Random Forest ($\$350$) violate the budget constraint. Naïve Bayes ($\$50$) and Logistic Regression ($\$70$) are cheaper, but sacrifice predictive accuracy ($84.8\%$ and $86.4\%$).
  • Scenario 2: Clinical Hospital Diagnostic AI with High Error Consequences:
    • Selected Model: Deep Learning (CNN).
    • Rationale: Maximizes accuracy ($97.8\%$), F1-score ($0.97$), and Information Gain ($0.95$), which is essential in clinical contexts where diagnostic errors can be fatal.
    • Why Others Fail: Cost and training duration are secondary in high-stakes clinical settings; choosing cheaper alternatives like Decision Trees or Naïve Bayes introduces unacceptable false negative and false positive rates.
  • Scenario 3: Predictive Maintenance with Hourly Retraining Cycle ($< 60\text{ min}$):
    • Selected Model: Random Forest.
    • Rationale: Delivers strong predictive performance ($94.3\%$ accuracy, $0.93$ F1-score) within an $18\text{-minute}$ training window, fitting comfortably into the $1\text{-hour}$ retraining budget.
    • Why Others Fail: Deep Learning ($90\text{ min}$) exceeds the hourly window. Decision Tree ($4\text{ min}$) fits easily, but sacrifices approximately $6\%$ in accuracy.
  • Scenario 4: Customer Segmentation over Millions of Unlabeled Records:
    • Selected Model: K-Means Clustering.
    • Rationale: Operates as an unsupervised clustering algorithm, making it suitable for unlabeled datasets. It has low computational cost ($\$60$) and fast execution ($5\text{ min}$).
    • Why Others Fail: Supervised algorithms (Decision Trees, CNN, XGBoost) require ground-truth target labels and cannot operate on unlabeled datasets.
  • Scenario 5: Embedded Edge Device Deployment with Constrained Memory:
    • Selected Model: Decision Tree.
    • Rationale: Operates with low memory overhead, trains in $4\text{ min}$, costs $\$80$, and delivers $88.5\%$ accuracy with simple if-then compilation.
    • Why Others Fail: Deep Learning, ANNs, and XGBoost exceed the memory and compute capacity of embedded edge chips. KNN is unsuited for edge deployment because it requires storing the entire training set in active memory for inference.

Data Analytics, Visual Communication, and Ethics in Business

• Ethics of Visual Distortion:

  • Truncated Axes: Truncating the vertical axis (Y-axis) on bar charts distorts visual proportions, exaggerating minor differences (e.g., making a $2\%$ growth look like a $200\%$ surge).
  • Ethical Boundary: Visualizations cross from ethical persuasion into manipulation when graphical choices lead non-expert audiences to draw conclusions contradicted by the raw data.
  • Reputational Risk: Organizations deploying deceptive dashboards risk regulatory scrutiny, loss of credibility, and diminished stakeholder trust.

• Exploratory Data Analysis: Correlation versus Causation:

  • Confounding Factors: Statistical correlation between two variables does not establish causal dependency. Confounding variables often drive observed relationships.
  • Supply Chain Anecdote: A retailer observing a strong correlation between warehouse shipping delays and local ice cream sales in Bangkok might mistake the two as causally linked. In reality, extreme seasonal heat simultaneously increases ice cream demand and strains transport cooling systems, leading to logistics bottlenecks.

• Operational Dashboards versus Strategic Narrative Reports:

  • Operational Dashboards: Provide real-time KPI visibility to help managers monitor active operations and address short-term bottlenecks.
  • Narrative Reports: Combine static visuals with qualitative context, strategic explanations, and forward-looking recommendations. Best suited for high-stakes decisions like board meetings, market expansion, and capital investments where live numbers alone lack needed context.

• Translating Qualitative Data into Analytical Metrics:

  • Qualitative Complexity: Subjective feedback (customer sentiment, brand perception, open-ended surveys) lacks native numerical structure.
  • Quantification Methods: Natural language processing (NLP), thematic coding, and sentiment polarity scoring ($-1.0$ to $+1.0$) transform subjective text into quantitative metrics.
  • Visual Presentation: Stacked bar charts (showing positive/neutral/negative proportions) and sentiment distribution plots are often more informative than unstructured word clouds.

• Cognitive Load and Gestalt Principles:

  • Shortcomings of 3D Visualizations: 3D pie charts introduce visual distortion—slices in the foreground appear artificially larger than those in the background due to perspective warping.
  • Cognitive Overload: Using excessive color palettes and unnecessary textures exhausts working memory.
  • Alternative Strategies: A sorted horizontal bar chart with clean baseline alignment, minimal labeling, and a single accent color to highlight key regions communicates regional revenue breakdowns more effectively.

• Data Governance and Cross-Border Compliance:

  • Regulatory Frameworks: The European Union's General Data Protection Regulation (GDPR) and Thailand's Personal Data Protection Act (PDPA) enforce strict guidelines on data privacy, consent, and user tracking.
  • Analytical Requirements: Analysts operating cross-border pipelines must implement data pseudonymization, aggregation, and role-based access before publishing customer dashboards.

• Role of Generative AI in Business Intelligence:

  • Automated Reporting: Generative AI tools can automate routine analytical tasks like writing queries, producing standard charts, and generating executive summaries.
  • Human Analysts' Value: Shifts analyst focus from manual visualization creation toward critical data validation, contextual interpretation, and ethical oversight.

• Handling Missing Data in Emerging Markets:

  • Treatment Strategies: Analysts address incomplete or delayed market data via row deletion, imputation (substituting mean, median, or predictive model values), or explicit missingness flags.
  • Visualizing Uncertainty: Visual indicators like dashed trendlines, shaded confidence bands, and missing-data heatmaps help managers assess data reliability and make informed risk calculations.

• Visualizing Financial and Sustainability Risks:

  • Financial Risk Visuals: Network graphs reveal transaction counterparty exposures and fraud patterns (e.g., circular invoice trails), while heatmaps display liquidity risk across currencies and payment deadlines.
  • ESG Reporting Framework: Environmental, Social, and Governance dashboards should provide separate views for carbon footprints (Scope 1 direct, Scope 2 electricity, Scope 3 supply chain), labor standards, and supplier audits, benchmarked against targets to reduce perceived greenwashing.

Applied Enterprise Analytics Case Studies

• Case Study 1: The Inventory Mirage at Siam Retail Logistics:

  • The Aggregation Trap: A high overall fulfillment metric ($95\%$) masked severe stockouts in fast-moving consumer categories because overstocked, slow-moving industrial items inflated the average.
  • Recommended Metrics: Shift focus to category-level stockout rates, inventory turnover velocity, regional availability heatmaps, and demand-to-stock ratios.
  • Dashboard Architecture: Implement a multi-pane layout separating overall inventory health, category stockout heatmaps, and reorder alerts for critical products.

• Case Study 2: Rebranding Bangkok Brews Product Launch:

  • Visual Flaws of Radar Charts: Multi-series radar charts obscure data when filled polygons overlap, concealing negative consumer feedback regarding price sensitivity and bitter taste.
  • Visual Redesign: Use stacked horizontal bar charts to illustrate positive, neutral, and negative sentiment across product attributes (taste, price, packaging), paired with a scatter plot comparing willingness-to-pay against retail prices.

• Case Study 3: The Expats' Dilemma – Regional Sales Disparities:

  • Contextual Blindness: Relying solely on gross sales made Thailand appear to be the primary market, overlooking Vietnam's rapid $35\%$ growth achieved with minimal advertising.
  • Resource Allocation Strategy: Plot marketing expenditure (X-axis) against revenue growth rate (Y-axis) in a scatter plot to identify high-efficiency growth markets that warrant greater investment.

• Case Study 4: Customer Churn at FinTech Siam:

  • Audience Misalignment: Presenting technical survival curves and a 50-variable correlation matrix overwhelmed board members without highlighting actionable causes.
  • Executive Redesign: Simplify reporting to focus on student churn rates before and after the introduction of a new transaction fee, using clear annotations to show how transaction fees drive account cancellations.