Data Mining: An Overview

Comprehensive vocabulary flashcards covering key definitions, algorithms, evaluation metrics, and foundational concepts from the lecture on Data Mining.

Data Mining

An interdisciplinary subfield of computer science representing the computational process of discovering patterns in large datasets involving methods at the intersection of artificial intelligence, machine learning, statistics, and database systems.

Knowledge Discovery in Databases (KDD)

The overall process of discovering knowledge from data, in which data mining serves as the core analysis step.

Data Fishing (Data Dredging)

A 1960s term used by statisticians to refer to what was considered the bad practice of analyzing data without an a-priori hypothesis.

Gregory Piatetsky-Shapiro

The researcher who coined the term 'Knowledge Discovery in Databases' for the first workshop on the topic (KDD-1989).

KDD-95

The first international conference on Data Mining and Knowledge Discovery held in Montreal in 1995 under AAAI sponsorship, co-chaired by Usama Fayyad and Ramasamy Uthurusamy.

Usama Fayyad

Co-chair of KDD-95 who launched the journal Data Mining and Knowledge Discovery in 1996 as its founding Editor-in-Chief and later started SIGKDD Explorations.

CRISP-DM

Standing for Cross Industry Standard Process for Data Mining, a six-phase model consisting of Business Understanding, Data Understanding, Data Preparation, Modeling, Evaluation, and Deployment.

Overfitting

A phenomenon where data mining algorithms find patterns in the training set that are not present in the wider general dataset, evaluated by testing learned patterns on an unseen test set.

Anomaly Detection

The identification of items, events, or observations in data mining that do not conform to an expected pattern or other items in a dataset; also referred to as outliers, novelties, noise, deviations, and exceptions.

Unsupervised Anomaly Detection

Anomaly detection techniques that detect anomalies in an unlabeled test dataset.

Supervised Anomaly Detection

Anomaly detection techniques that require a dataset labeled as 'normal' and 'abnormal'.

Semi-Supervised Anomaly Detection

Anomaly detection techniques that construct a model representing normal behavior to identify deviations.

Dorothy Denning

The researcher who proposed using anomaly detection for intrusion detection systems (IDS) in 1986.

Misuse Detection

The counterpart of anomaly detection in intrusion detection systems (IDS), which relies on signatures of known attacks.

Association Rule Learning

A rule-based machine learning method for discovering interesting relations between variables in large databases using measures of interestingness.

Support (Association Rules)

An indication of how frequently an item-set $X$ appears in the database, defined as $\text{supp}(X) = \frac{|X|}{N}$.

Confidence (Association Rules)

An indication of how often a rule $X \rightarrow Y$ is found to be true, defined as $\text{conf}(X \rightarrow Y) = \frac{\text{supp}(X \times Y)}{\text{supp}(X)}$.

Apriori Algorithm

An association rule learning algorithm that uses a breadth-first search strategy to find frequent itemsets.

Eclat Algorithm

An association rule learning algorithm that uses set intersection to identify frequent itemsets.

FP-growth Algorithm

An association rule learning algorithm (standing for Frequent Pattern) that mines frequent itemsets without candidate generation.

Cluster Analysis

The task of grouping a set of objects in such a way that objects in the same group (cluster) are more similar to each other than to those in other groups.

Centroid-Based Clustering

A clustering model where clusters are represented by a central vector (such as in $k$-means clustering), which may not necessarily be a member of the dataset.

Density-Based Clustering

A clustering model where clusters are defined as areas of higher density than the remainder of the dataset, regarding sparse separating areas as noise and border points.

Davies-Bouldin Index

An internal evaluation metric for clustering defined as DB = \frac{1}{n} \times \text{max}_{j \neq i} \times \frac{\frac{\text{\rho}_i + \text{\rho}_j}{d(c_i, c_j)}}{1}.

Dunn Index

An internal clustering evaluation metric that aims to identify dense and well-separated clusters.

Silhouette Coefficient

An internal evaluation metric for clustering that contrasts the average distance to elements in the same cluster with the average distance to elements in other clusters.

Rand Measure

An external clustering evaluation metric that computes how similar generated clusters are to benchmark classifications, defined as $RI = \frac{TP + TN}{TP + FP + FN + TN}$.

Jaccard Index

An external evaluation measure used to quantify the similarity between two datasets or clusterings, calculated as $J(A, B) = \frac{|A \times B|}{|A \times B|} = \frac{TP}{TP + FP + FN}$.

Statistical Classification

The problem in machine learning and statistics of identifying to which of a set of categories a new observation belongs, based on a training set of known category membership.

Fisher's Linear Discriminant Function

An early classification rule developed by Fisher for two-group problems under the assumption of a multivariate normal distribution within groups.

Feature Vector

An $n$-dimensional vector of measurable individual properties (explanatory or independent variables) used to describe an instance for classification.

Linear Predictor Function

A score function for linear classifiers of the general form \text{score}(\text{X}_i, k) = \text{\beta}_k \times \text{X}_i, combining a feature vector with a weight vector using a dot product.

No-Free-Lunch Theorem

A phenomenon explaining that there is no single classifier that works best on all given problems.

Regression Analysis

A statistical process for estimating relationships among variables, focusing on the relationship between a dependent variable and one or more independent predictor variables.

Method of Least Squares

The earliest form of regression published by Legendre in 1805 and Gauss in 1809 to determine the orbits of solar bodies from astronomical observations.

Homoscedasticity

The classical assumption in regression analysis that the variance of the error term is constant across all observations.

Interpolation vs. Extrapolation

Interpolation refers to predicting a value within the range of data values used for model fitting, whereas extrapolation refers to predicting outside that range.

Automatic Summarization

The process of reducing a text document with a computer program in order to create a summary that retains the most important points of the original document.

Extraction-Based Summarization

A summarization method where the system extracts objects (words, keyphrases, or whole sentences) directly from the collection without modifying them.

Abstraction-Based Summarization

A summarization method that involves paraphrasing sections of the source document using natural language generation technology.

ROUGE

Standing for Recall-Oriented Understudy for Gisting Evaluation, a recall-based measure determining how well a machine summary covers content present in human reference summaries.

TextRank

An unsupervised graph-based ranking algorithm for keyphrase extraction and sentence summarization that exploits text structure using PageRank principles.

LexRank

An unsupervised text summarization algorithm that constructs a graph using cosine similarity of TF-IDF vectors of sentences and applies PageRank to rank them.

Maximal Marginal Relevance (MMR)

A method used in multi-document extractive summarization to eliminate information redundancy and ensure diverse results.

Submodular Functions

Set functions that naturally model notions of coverage, information, representation, and diversity in data summarization problems.