AI midterm

Linear Regression

Models the relationship between a dependent variable (y) and independent variables (x1, x2,..., xn).

Residuals

The differences between observed values and predicted values (ei = yi - ŷi).

Sum of Squared Errors (SSE)

Objective function that minimizes the total squared errors in a regression model.

Mean Squared Error (MSE)

The average of the squared errors; measures the quality of a regression model.

Gradient Descent

An optimization algorithm used to minimize the loss function by iteratively adjusting model parameters.

Learning Rate (α)

A parameter indicating how much to change the model in response to the estimated error.

k-Fold Cross-Validation

assess the generalization of a model by dividing the dataset into k parts.

Ridge Regression

A regression method that adds L2 penalty to the loss function, reducing the magnitude of the coefficients.

Lasso Regression

A regression method that adds L1 penalty to the loss function, allowing for coefficient shrinkage and feature selection.

Elastic Net

A regression technique that combines both L1 and L2 penalties for regularization.

Logistic Regression

A method for binary classification predicting the probability of binary outcomes.

Sigmoid Function

A function that maps any real-valued number into the (0, 1) interval, used to model probabilities.

Log-Odds

The logarithm of the odds, representing the likelihood of an event occurring.

Maximum Likelihood Estimation (MLE)

A method used in statistics to estimate the parameters of a statistical model.

Gradient Ascent

An optimization algorithm that updates parameters to maximize the log-likelihood function.

One-vs-Rest (OvR) Approach

A method where multiple binary classifiers are trained to distinguish one class from the rest.

Softmax Function

A function that converts raw scores into probabilities that sum to one in multi-class classification.

Confusion Matrix

A tabular layout that allows visualization of the performance of an algorithm.

Accuracy

The ratio of correctly predicted instances to the total instances.

Precision

The ratio of true positives to the total predicted positives.

Recall

The ratio of true positives to the total actual positives.

F1-Score

The harmonic mean of precision and recall, balancing both metrics.

One-Hot Encoding

A method for transforming categorical variables into a binary matrix representation.

Decision Trees

A model that makes decisions based on a tree-like structure of nodes representing feature splits.

Gini Index

A measure of impurity used in decision trees to create splits that minimize impurity.

Entropy

A measure of disorder or uncertainty used in decision trees to create splits.

Overfitting

When a model learns noise and details in the training data to the extent that it negatively impacts performance on new data.

Pruning

The process of removing parts of a tree that do not provide significant power in prediction.

Random Forests

An ensemble learning method that constructs multiple decision trees for improved accuracy.

Bootstrap Samples

Random sampling of data with replacement to create subsets for each decision tree in a random forest.

Out-of-Bag (OOB) Error

An estimate of the model's generalization performance based on data points not included in the bootstrap sample.

Threshold

A value that separates predicted outcomes in classification algorithms.

Hyperplane

A subspace of one dimension less than its ambient space, used in regression and classification.

L2 Penalty

A regularization term that helps prevent overfitting by penalizing large coefficients.

L1 Penalty

A regularization term that can shrink some coefficients to zero for feature selection.

Feature Selection

The process of selecting a subset of relevant features for model training.

Decision Boundary

The surface that separates instances of different classes in classification problems.

Collinearity

A situation in which some predictor variables are highly correlated, potentially complicating modeling.

Feature Engineering

The process of using domain knowledge to create features that make machine learning algorithms work.

Model Generalization

The ability of a model to perform well on unseen data.

Numerical Stability

The property of an algorithm to avoid significant numerical errors during computations.

Heuristic

A rule of thumb or strategy to simplify decision-making processes.

ID3 Algorithm

An algorithm used to construct decision trees based on information gain.

Pruning Techniques

Methods to reduce the complexity of decision trees by cutting back on branches.

Bagging

An ensemble method that improves the stability and accuracy of machine learning algorithms through averaging.

Top-Down Approach

Building a decision tree by starting with the whole dataset and splitting it down.

Binary Classification

Classifying instances into one of two distinct classes.

Model Evaluation

The process of assessing how well your machine learning model performs.

Training Set

The dataset used to train a machine learning model.

Test Set

The dataset used to evaluate the performance of a trained machine learning model.

Multicollinearity

Occurs when two or more predictor variables in a regression model are highly correlated.

=