1/77
Comprehensive vocabulary flashcards covering statistical modeling, A/B testing methodologies, machine learning fundamentals, ensemble methods, time series forecasting, and Spark/Databricks architecture.
Assumptions of OLS Linear Regression
Key conditions required for Ordinary Least Squares regression: linearity in parameters, independent errors, constant error variance (homoscedasticity), no perfect multicollinearity, errors with mean zero and uncorrelated with predictors (exogeneity), and normal errors (required only for small-sample inference).
Exogeneity Violation
A condition in linear regression where errors are correlated with predictor variables (X); it is considered the most severe violation because it introduces bias into coefficient estimates, unlike most other violations which only distort standard errors.
Heteroscedasticity
The violation of the constant error variance assumption; coefficients remain unbiased, but standard errors, t-statistics, and p-values become incorrect (typically overly optimistic).
Multicollinearity
A condition where predictor variables are highly correlated, causing coefficient estimates to exhibit inflated variance and unstable signs, though model predictions may remain accurate.
Variance Inflation Factor (VIF)
A diagnostic metric used to detect multicollinearity among predictors, where values exceeding 5 to 10 indicate problematic correlation.
Clustered Standard Errors
A correction applied to standard errors when errors are correlated within groups or clusters (such as repeated measures per workspace) to prevent understating standard errors and overclaiming statistical significance.
Omitted Variable Bias
The bias introduced into an included predictor's coefficient estimate when an excluded variable both influences the outcome y and is correlated with that included predictor.
Elasticity (Log-Log Regression)
The interpretation of coefficient β when both x and y are log-transformed, signifying that a 1% increase in x is associated with approximately a β% change in y.
R2 (Coefficient of Determination)
The proportion of variance in the outcome variable y explained by the regression model; it never decreases upon adding more features and provides no indication of causality or out-of-sample accuracy.
Lasso Regression (L1)
A regularized regression technique that applies an absolute-value penalty on coefficient magnitudes, capable of shrinking coefficients to exactly zero to perform automated feature selection.
Ridge Regression (L2)
A regularized regression technique that applies a squared-magnitude penalty, shrinking all coefficients smoothly toward zero while stably handling highly correlated predictors.
Logistic Regression
A classification model that models the log-odds of a binary outcome as a linear combination of predictors, where a 1-unit increase in a feature multiplies the odds of the outcome by eβ.
Interaction Term
A predictor formed by multiplying two features (x1×x2), allowing the effect of x1 on the target to depend on the value of x2.
Cook's Distance / Leverage
Regression diagnostic measures used to detect influential outlier data points that exert disproportionate pull on model coefficients.
Generalized Linear Model (GLM)
A flexible framework connecting a linear predictor to the mean of an exponential-family outcome via a link function (e.g., logistic link for binary outcomes, Poisson for event counts, Gamma for positive skewed data).
Parallel Trends Assumption
The core identifying assumption of difference-in-differences analysis stating that, in the absence of treatment, the treatment and control groups would have followed parallel trajectory paths over time.
Regularization
The addition of a mathematical penalty on model complexity (coefficient magnitude) to trade a small increase in bias for a substantial reduction in variance.
p-value
The probability of observing sample test data at least as extreme as the observed data under the assumption that the null hypothesis is true; it is not the probability that the null hypothesis itself is true.
Type I Error
A false positive error occurring when a true null hypothesis is incorrectly rejected, defined at the significance rate α.
Type II Error
A false negative error occurring when the test fails to reject a false null hypothesis (missing a genuine effect), defined at rate β.
Statistical Power
The probability (1−β) of correctly detecting a true effect of a specified magnitude; it increases with larger sample size, larger effect size, and higher α, and decreases with higher variance.
95% Confidence Interval
An interval estimate such that if an experiment were repeated many times independently, 95% of intervals constructed using this procedure would contain the true population parameter.
Central Limit Theorem (CLT)
The statistical principle that the sample mean of many independent, identically distributed variables with finite variance approaches a normal distribution as sample size grows, regardless of the underlying shape of the distribution.
Bessel's Correction
Dividing by n−1 rather than n in the sample variance calculation to correct for the degree of freedom lost when estimating the sample mean, ensuring an unbiased variance estimate.
Welch's t-test
A two-sample hypothesis test for comparing means that does not assume equal population variances, serving as the safer default over Student's t-test.
Chi-Square Test
A statistical hypothesis test used to assess associations between two categorical variables or compare observed cell counts against expected counts, assuming expected counts are at least roughly 5.
Mann-Whitney U Test
A non-parametric statistical test that compares the rank distributions of two independent groups, useful when the metric is ordinal or heavily skewed and normal distribution assumptions fail.
Bootstrapping
A non-parametric statistical technique that repeatedly resamples a single dataset with replacement to simulate the sampling distribution of a statistic (e.g., confidence intervals for medians or ratio metrics).
Bayes' Theorem
A mathematical formula for calculating conditional probability: P(A∣B)=P(B)P(B∣A)P(A), showing how prior probabilities are updated based on observed evidence.
Law of Large Numbers (LLN)
A mathematical theorem stating that as sample size increases, the empirical sample mean converges asymptotically toward the true population expected value.
Simpson's Paradox
A statistical phenomenon wherein a consistent trend observed across individual subgroups disappears or reverses when the groups are aggregated together due to confounding subgroup weights.
Survivorship Bias
A form of selection bias where conclusions are distorted because the analysis focuses exclusively on entities that passed an attrition filter (such as analyzing only retained customers to study renewal drivers).
Standard Error (SE)
The standard deviation of an estimate's sampling distribution (e.g., SE=nSD), describing the precision of the estimated statistic rather than individual data dispersion.
Maximum Likelihood Estimation (MLE)
A method of estimating statistical model parameters by finding parameter values that maximize the likelihood of having generated the observed data.
Minimum Detectable Effect (MDE)
The smallest genuine effect size an A/B test has sufficient statistical power to reliably detect at pre-specified α and power thresholds, typically determined by business viability.
SUTVA Violation (Interference)
A violation of the Stable Unit Treatment Value Assumption occurring when the treatment assignment of one unit influences the outcome of another unit (e.g., users sharing clusters or workspaces), typically biasing effect estimates toward zero.
Peeking Problem
The practice of repeatedly inspecting experimental results during data collection and stopping the test when p<0.05, which inflates the true false positive rate far beyond nominal levels.
CUPED
Controlled-experiment Using Pre-Existing Data: an experimental variance reduction technique that subtracts a scaled pre-experiment covariate (Yadj=Y−θ(X−Xˉ)) to yield higher statistical power with the same expected effect.
Sample Ratio Mismatch (SRM)
A statistically significant deviation between observed sample allocation counts and intended experimental design ratios, serving as a critical diagnostic indicator of assignment or telemetry bugs.
Novelty Effect
An experimental phenomenon where users engage more intensely with a modified feature purely because it is unfamiliar, resulting in an artificial metric lift that fades over time.
Primacy Effect
Initial user resistance or diminished engagement observed when introducing a change due to habituation to the old experience, which diminishes as users adapt.
Guardrail Metrics
Critical operational or business health indicators (such as query latency p95, error rates, compute costs, or retention) that must remain uncompromised even if the primary metric improves.
A/A Test
An experiment where both treatment and control groups receive the exact same experience to validate randomization pipelines, telemetry, and ensure the empirical false positive rate conforms to α.
Goodhart's Law
The adage that when a measure becomes a target, it ceases to be a good measure, as optimization efforts focus on gaming the proxy rather than enhancing the underlying goal.
Bias-Variance Trade-off
The decomposition of expected prediction error into bias2+variance+irreducible noise, highlighting how under-parameterized models underfit (high bias) while overly complex models overfit (high variance).
Data Leakage
The inadvertent introduction of information into the training dataset that would not be available at the actual time of prediction, artificially inflating validation performance.
Precision
The proportion of predicted positive instances that are truly positive: TP+FPTP, measuring prediction accuracy on positive calls.
Recall
The proportion of actual positive instances correctly captured by the classifier: TP+FNTP, measuring sensitivity.
ROC-AUC
Area Under the Receiver Operating Characteristic curve; quantifies ranking performance across all decision thresholds and equals the probability that a random positive instance is ranked above a random negative one.
PR-AUC
Area Under the Precision-Recall curve; a classification metric focusing exclusively on the positive class, providing greater informativeness than ROC-AUC when positive outcomes are rare.
Model Calibration
The property where predicted class probabilities correspond directly to observed empirical probabilities (e.g., among samples scored at 0.3, roughly 30% are true positives).
Target Encoding
A categorical encoding method replacing categories with the average target value; it poses severe data leakage risks unless calculated strictly out-of-fold with smoothing.
Principal Component Analysis (PCA)
An unsupervised linear dimensionality reduction technique that projects correlated features onto orthogonal directions of maximum variance.
k-means Clustering
An unsupervised clustering algorithm that iteratively assigns observations to the closest of k centroids to minimize within-cluster sum of squared distances.
Generative vs Discriminative Models
Generative models learn the joint probability distribution P(x,y) (e.g., Naive Bayes), whereas discriminative models learn the conditional distribution P(y∣x) or decision boundary directly (e.g., logistic regression).
Gradient Descent
An iterative optimization algorithm that updates model parameters in the direction opposite to the gradient of the loss function to locate a minimum.
Bagging (Bootstrap Aggregating)
An ensemble technique that trains multiple independent models in parallel on bootstrap resamples of training data and averages their predictions to decrease variance.
Out-of-Bag (OOB) Error
The evaluation error computed on the approximately 37% of training instances not sampled in an individual tree's bootstrap sample (1−e1≈0.632 sampled), providing a built-in validation metric.
Permutation Feature Importance
A model-agnostic feature importance metric that measures the drop in validation performance after randomly shuffling the values of a single feature.
SHAP (Shapley Additive exPlanations)
A game-theoretic framework that computes Shapley values to attribute the contribution of each individual feature to a specific model prediction both locally and globally.
Stationarity
A time-series property where the mean, variance, and autocorrelation structure remain constant over time, serving as a core prerequisite for ARIMA modeling.
ARIMA(p, d, q)
A time-series forecasting model combining p autoregressive lag terms, d orders of differencing to induce stationarity, and q moving-average lagged forecast error terms.
Autocorrelation
The correlation of a time series with lagged values of itself, which violates independence assumptions and causes standard errors from naive tests to be understated.
Interrupted Time Series Analysis
A quasi-experimental methodology that evaluates the impact of an intervention by comparing post-intervention trends against a modeled pre-intervention baseline.
Exponential Smoothing (ETS)
A time-series forecasting method that generates predictions using weighted averages of past observations with exponentially decaying weights, with options for trend and seasonal adjustments.
Databricks Lakehouse
A unified data platform architecture merging the low-cost, open-format storage of data lakes with the ACID transactions, data governance, and high query performance of data warehouses.
Delta Lake
An open storage layer built on Parquet file formats using a file-based transaction log to deliver ACID transactions, schema enforcement and evolution, time travel, and upsert/MERGE operations.
Medallion Architecture
A data organization design pattern comprised of three layers: Bronze (raw ingested data), Silver (cleaned, transformed, deduplicated data), and Gold (aggregated, curated business-level tables for BI and ML).
Unity Catalog
Databricks' centralized data governance framework providing access control, data lineage tracking, auditing, and discovery across data and AI assets under a three-level namespace (catalog.schema.table).
MLflow
An open-source ML lifecycle management platform supporting experiment tracking (logging parameters and metrics), model packaging, centralized model registry, and production deployment.
Spark Lazy Evaluation
An execution paradigm where transformations (such as select or filter) only construct a logical execution plan, delaying computation until an action (such as count or write) triggers optimized physical execution.
Spark Shuffle
The physical movement and redistribution of data across cluster worker nodes over the network, triggered by wide transformations such as groupBy and join, incurring heavy disk and network I/O.
Broadcast Join
A distributed join optimization where Spark copies a small lookup table to the memory of every worker executor, enabling large tables to be joined locally without network shuffling.
Data Skew (Spark)
An uneven distribution of partition keys across a cluster where a few tasks process disproportionately huge volumes of data, causing long-tail execution bottlenecks.
Databricks Unit (DBU)
A normalized unit of compute processing capability per hour utilized as the fundamental unit of usage-based billing across Databricks platform products and SKUs.
Mosaic AI Agent Evaluation
Databricks' evaluation framework designed for assessing generative AI applications and autonomous agents using automated LLM judges to benchmark quality and safety.
Complete Separation (Perfect Separation)
A condition in logistic regression where a predictor perfectly separates positive and negative classes, preventing the existence of finite maximum likelihood estimates as coefficients trend to infinity.
Linear Probability Model (LPM)
An Ordinary Least Squares regression applied to a binary outcome that estimates percentage-point marginal effects directly, though it can yield predictions outside [0,1] and inherently violates homoscedasticity.