Data Analysis Module 6-10

AmA university

Generalized Linear Model (GLM)

Extends linear regression to allow a non-normal dependent variable distribution and non-linear relationships; made of a random component, systematic component, and link function.

GLM: Random Component

Represents the distribution of the dependent variable (e.g. normal, binomial, Poisson, gamma).

GLM: Link Function

Mathematical function connecting the linear predictor to the mean of the random component (e.g. identity, log, logit).

Logistic Regression

A GLM used for binary outcomes; uses the logit link function and Bernoulli/binomial distribution (e.g. predicting churn).

Poisson Regression

A GLM used to model count data (e.g. number of defects); uses the log link function and Poisson distribution.

Maximum Likelihood Estimation (MLE)

A method that finds parameter values making the observed data most likely to occur, by maximizing the likelihood function.

Likelihood Function

The probability of observing the data given specific model parameters; MLE maximizes this function to estimate parameters.

Overfitting

When a model is too complex and learns noise in training data instead of the underlying pattern, hurting performance on new data.

Ridge Regression (L2)

Regularization technique that penalizes the sum of squared coefficients; shrinks weights but never sets them to zero.

Lasso Regression (L1)

Regularization technique that penalizes the sum of absolute coefficient values; can shrink some weights to exactly zero (feature selection).

Elastic Net Regression

Hybrid of Ridge and Lasso; combines L1 and L2 penalties to shrink coefficients and perform feature selection.

Cross-Validation

A resampling technique that repeatedly splits data into training/testing sets to more reliably evaluate model performance.

K-fold Cross-Validation

Divides the dataset into k folds, trains on k-1 folds, and tests on the remaining fold, rotating through all folds.

Common regression metrics

MSE (avg squared error), MAE (avg absolute error), R² (variance explained), and MAPE (avg % error).

Generalized Additive Model (GAM)

Extends linear regression by modeling the response as a sum of smooth (non-linear) functions of predictors instead of assuming linearity.

Unsupervised Learning

Machine learning that finds hidden patterns or structures in unlabeled data, without predefined targets or outputs.

Supervised vs. Unsupervised Learning

Supervised predicts outputs from labeled data; unsupervised explores unlabeled data to discover structure with no predefined targets.

Curse of Dimensionality

As feature count increases, data becomes sparse and distance metrics lose meaning, hurting many ML algorithms' performance.

Dimensionality Reduction: 2 approaches

Feature selection (choosing a subset of existing features) and feature extraction (transforming features into new, compact ones).

Principal Component Analysis (PCA)

A feature extraction technique that projects data onto orthogonal components ordered by variance, keeping the top components.

t-SNE

A non-linear dimensionality reduction technique that preserves local structure, mainly used for visualizing high-dimensional data.

K-means Clustering

A clustering algorithm that groups data into k clusters based on distance to cluster centroids (e.g. customer segmentation).

Hierarchical Clustering

Clustering that builds a tree (dendrogram) of nested clusters — agglomerative (bottom-up merging) or divisive (top-down splitting).

DBSCAN

Density-Based Spatial Clustering; groups points into clusters based on density (core points + neighbors within radius ε), and flags outliers as noise.

Silhouette Coefficient

A cluster validation metric (range -1 to 1) measuring how well-separated and cohesive clusters are; higher is better.

Time Series

A sequence of data points measured at regular time intervals, used to identify patterns and forecast future values.

4 components of a time series

Trend, seasonality, cyclicity, and residual (irregular/random variation).

Trend (time series)

The overall long-term direction of a time series — increasing, decreasing, or stationary.

Seasonality

Regular, fixed-interval fluctuations in a time series (e.g. daily, monthly, yearly patterns).

Autocorrelation

The correlation between a time series and a lagged (delayed) version of itself; helps detect repeating patterns.

Stationarity

A time series whose statistical properties (mean, variance) stay constant over time, making it easier to model.

Moving Average (smoothing)

Smooths a time series by averaging a fixed window of past values to reduce short-term noise.

Exponential Smoothing

Forecasting method that weights recent observations more heavily than older ones; includes SES, Holt's, and Holt-Winters' methods.

Seasonal Decomposition: additive vs multiplicative

Additive suits data with constant variance (components summed); multiplicative suits data with variance that changes over time (components multiplied).

ARIMA

AutoRegressive Integrated Moving Average; combines AR (past values), I (differencing for stationarity), and MA (past errors) to forecast time series.

ARIMA(p, d, q)

Notation for ARIMA parameters: p = autoregressive terms, d = degree of differencing, q = moving average terms.

Univariate vs. Multivariate Time Series

Univariate tracks a single variable over time; multivariate tracks multiple variables and their relationships over time.

Recurrent Neural Network (RNN)

A neural network designed for sequential data; feeds previous output back into the network to capture temporal dependencies.

Vanishing Gradient Problem

Issue in simple RNNs where gradients shrink during training, making it hard to learn long-term dependencies.

LSTM (Long Short-Term Memory)

An RNN variant with memory cells and input/output/forget gates that solves the vanishing gradient problem for long-term dependencies.

LSTM: Forget Gate

The gate that decides what information to discard from the memory cell.

Time series preprocessing steps

Handling missing values, normalization/scaling, and feature engineering (e.g. lagging, differencing).

Ensemble methods (forecasting)

Combine predictions from multiple models to improve accuracy; includes bagging, boosting, and stacking.

Attention Mechanism

A neural network technique that lets the model weigh and focus on the most relevant parts of an input sequence when predicting.

Gated Recurrent Unit (GRU)

An RNN variant similar to LSTM but with fewer parameters, making it faster to train.

Naive Forecasting

The simplest forecasting method — uses the most recent observed value as the forecast for the next period.

Big Data: the 5 V's

Volume (amount), Velocity (speed), Variety (data types), Veracity (accuracy/reliability), and Value (usefulness of insights).

NoSQL Database

A non-relational database designed for flexibility, scalability, and handling large or unstructured datasets (vs. fixed-schema relational DBs).

4 types of NoSQL databases

Key-Value stores (Redis), Document stores (MongoDB), Column-Family stores (Cassandra), and Graph databases (Neo4j).

Denormalization (NoSQL)

Intentionally storing redundant data within a document to reduce the number of queries needed and speed up retrieval.