Data Mining Exam

Bootstrapping

Resampling method with replacement used to estimate statistics and assess model stability when sample size is limited

Sample Size Rule of Thumb

Requires at least 10 observations per input variable in the training set to ensure sufficient data for modeling

Transactional Data Rollup

Transforming long format transaction data to wide format with one row per observation using summary statistics like mean and sum

Missing Value Flag

Binary indicator variable created alongside imputed missing values to capture potential informative missingness

Equal Width Binning

Discretizing continuous variables into intervals that all have identical width while observation counts per bin vary

Equal Depth Binning

Discretizing continuous variables into percentile intervals so that every bin contains the exact same number of observations

Supervised Binning

Optimally categorizing numeric variables by utilizing target variable information to create maximally pure target class splits

Bonferroni Correction

Statistical adjustment for multiple testing that controls the Family-Wise Error Rate by dividing alpha by the number of tests

Family-Wise Error Rate

Probability of making one or more Type I errors across an entire family of statistical hypothesis tests

False Discovery Rate

Expected proportion of incorrect rejections among all rejected null hypotheses controlling false positives without being overly conservative

Antecedent and Consequent

In association rule A implies B itemset A is the antecedent condition and itemset B is the consequent result

Support

Probability P(A and B) measuring how frequently a combined itemset or association rule appears across all transactions

Confidence

Conditional probability P(B given A) measuring the percentage of transactions containing A that also contain B

Lift

Ratio of rule confidence to expected baseline confidence where values greater than 1 indicate a strong positive association

Apriori Algorithm

Two-step association analysis technique that generates frequent itemsets meeting minimum support then derives rules meeting minimum confidence

Root Node

Topmost node in a decision tree representing the full initial training dataset before any splits are applied

Leaf Node

Terminal node in a decision tree that contains final predicted class probabilities or response values and undergoes no further splits

Node Purity

Degree of homogeneity within a decision tree node regarding target variable classes where pure nodes contain only one class

Gini Impurity

Measure of node impurity calculated as 1 minus the sum of squared class probabilities where lower values indicate higher purity

Entropy

Information theory metric measuring node impurity based on negative sum of class probabilities times log base 2 of probabilities

Information Gain

Reduction in impurity achieved by splitting a parent node into child nodes used to select the optimal predictor split

Categorical Predictor Splitting

Grouping discrete factor levels into two optimal subsets to form binary candidate splits in decision tree algorithms

Prepruning

Stopping tree growth early during training by enforcing thresholds such as minimum gain or minimum node size

Post-pruning

Growing a full decision tree first then cutting back branches with low gain using cross-validation error rules like 1-SE

CART Algorithm

Classification and Regression Trees framework that uses greedy binary recursive partitioning based on Gini or SSE reduction

Conditional Trees

Decision tree approach using statistical hypothesis testing with p-value adjustments to select variables before finding optimal splits

Hard versus Fuzzy Clustering

Hard clustering assigns each object strictly to one cluster whereas fuzzy clustering assigns fractional membership probabilities

Hierarchical versus Flat Clustering

Hierarchical clustering builds nested tree structures like dendrograms while flat methods like K-means partition data directly into k groups

K-Means Clustering

Iterative partitioning algorithm that minimizes sum of squared Euclidean distances from data points to their assigned cluster centroids

Elbow Method

Technique for selecting optimal number of clusters k by finding the point where additional clusters yield diminishing returns in SSE

Euclidean Distance

Straight-line distance between two points in Euclidean space calculated as the square root of sum of squared differences

Manhattan Distance

Grid distance between two points calculated as the sum of absolute differences across all coordinates

Single Linkage

Hierarchical clustering distance metric defined by the minimum distance between any single point in cluster A and point in cluster B

Complete Linkage

Hierarchical clustering distance metric defined by the maximum distance between any point in cluster A and point in cluster B

Average Linkage

Hierarchical clustering linkage defined as the mean distance across all possible pairwise combinations of points between two clusters

Ward Linkage

Hierarchical clustering agglomerative method that merges cluster pairs to minimize total within-cluster sum of squared errors

DBSCAN

Density-based spatial clustering algorithm that groups densely packed points within distance epsilon and identifies sparse points as noise

Variable Clustering

Dimension reduction technique that groups related variables based on eigenvalues to eliminate redundancy and multicollinearity

K-Nearest Neighbors

Non-parametric instance-based learning algorithm that classifies or predicts a new sample based on majority vote or average of k nearest samples

Choosing K in k-NN

Small k values produce high variance models prone to overfitting whereas large k values create high bias models prone to underfittin