MACHINE LEARNING

Machine Learning

Teaching computers to learn patterns from data and make predictions.

Difference of ML from traditional programming

Traditional code follows written rules; ML learns patterns from examples.

Dataset

A collection of data used to train or evaluate a model.

Sample

One individual example or row in a dataset.

Feature

An input variable used for prediction. Example: house size.

Label or target

The correct output the model should predict. Example: house price.

data preprocessing

Cleaning and preparing data before training a model.

feature engineering

Creating or transforming features to help a model learn.

ML model

A mathematical system that learns patterns and makes predictions.

model parameters

Internal values that a model learns during training.

weight

A learned value controlling how strongly an input affects a prediction.

bias in a model equation

A learned constant that shifts the prediction up or down.

basic linear prediction formula

ŷ = wx + b, where w is the weight and b is the bias.

training

The process of learning and adjusting a model using data.

inference

Using a trained model to predict an output for new data.

supervised learning

Learning from labeled data with known correct answers.

unsupervised learning

Finding patterns in data without known labels.

clustering

Grouping similar data points together without predefined labels.

regression

Predicting a numerical value

classification

Predicting a category

loss function

A function measuring how wrong a model's predictions are.

Mean Squared Error (MSE)

The average of squared prediction errors.

gradient

The direction and rate at which the loss changes.

gradient descent

An optimization method that updates parameters to reduce loss.

learning rate

The step size used when updating model parameters.

if the learning rate is too high

Training may overshoot the minimum and fail to converge.

batch

A smaller group of training examples processed together.

epoch

One complete pass through the training dataset.

training set used for

Learning and updating the model's parameters.

validation set used for

Checking performance and tuning the model during development.

test set used for

Final evaluation on unseen data.

overfitting

Learning training data too closely and performing poorly on new data.

data leakage

When unavailable or test information improperly enters training.

generalization

A model's ability to predict accurately on new, unseen data.

neural network

A model using connected layers of artificial neurons to learn patterns.

artificial neuron

Combines weighted inputs and bias, then applies an activation.

hidden layer

An internal neural-network layer that processes information.

activation function

A function introducing nonlinearity so networks learn complex patterns.

ReLU

Outputs zero for negative inputs; keeps positive inputs unchanged.

forward pass

Sending inputs through a network to produce a prediction.

backpropagation

Calculating gradients backward through a neural network.

deep learning

Machine learning using neural networks with many layers.

Mean Absolute Error (MAE)

The average absolute difference between predictions and actual values.

Root Mean Squared Error (RMSE)

The square root of MSE; measures error in the original units.

accuracy

The percentage of predictions that are correct.

precision

Of predicted positives, the proportion that are truly positive.

recall

Of actual positives, the proportion the model correctly finds.

F1-score

The mean of precision and recall.

attention in a transformer

A mechanism that helps the model focus on relevant input information.

multimodal AI

AI that processes multiple data types, such as text, images, and audio.

AI agent

A system that uses a model and tools to take actions toward a goal.

Why was AlexNet important in 2012?

It demonstrated powerful GPU-based deep learning for image recognition.

What was the 1956 Dartmouth workshop important for?

It helped establish artificial intelligence as a research field.

Linear Regression

Predicts numerical values by fitting a linear relationship.

Logistic Regression

Predicts probabilities for classification, often between two classes.

Decision Tree

Makes predictions by following a series of decision rules.

Random Forest

Combines many decision trees to make more reliable predictions.

Gradient Boosting

Builds models sequentially, each correcting earlier prediction errors.

Support Vector Machine (SVM)

Finds a boundary that best separates classes, usually maximizing the margin.

K-Nearest Neighbors (KNN)

Predicts using the labels or values of nearby data points.

Naive Bayes

A probabilistic classifier assuming features are conditionally independent.

K-Means Clustering

Divides unlabeled data into K groups around cluster centers.

Hierarchical Clustering

Creates a tree of nested groups based on data similarity.

Principal Component Analysis (PCA)

Reduces dimensions while keeping as much data variation as possible.

Artificial Neural Network (ANN)

Connected artificial neurons that learn relationships from data.

Convolutional Neural Network (CNN)

A neural network that detects spatial patterns, especially in images.

Recurrent Neural Network (RNN)

A network that uses previous information when processing sequences.

Long Short-Term Memory (LSTM)

An RNN architecture designed to remember information over long sequences.

Autoencoder

A network that compresses data and learns to reconstruct it.

Generative Adversarial Network (GAN)

Two networks compete: one generates samples, the other detects fakes.

Transformer

A neural-network architecture using attention to process relationships in data.

Large Language Model (LLM)

A large model trained on text to understand and generate language.

Diffusion Model

A generative model that learns to create data by removing noise.

Reinforcement Learning (RL)

Learning actions through rewards and penalties from interactions.

helps a decision tree choose classification splits

Gini impurity or information gain.

Gini impurity of 0

All samples in the node belong to the same class.

algorithm that uses Euclidean distance to find neighbors

K-Nearest Neighbors (KNN).

Why can high accuracy be misleading with imbalanced classes?

A model can predict the majority class and miss rare cases.

A model predicts all transactions as normal in 98% normal data. Accuracy?

98%, even though it detects no fraud.

recall,

Of all actual positives, how many the model correctly detects.

high recall particularly important

When missing a positive case is costly, such as disease detection.

precision

Of predicted positives, how many are truly positive.

F1-score

Precision and recall into one balanced score.

Why does KNN need feature scaling?

To stop features with large numerical ranges dominating distances.

KNN uses age (20–80) and income (20,000–100,000). What is the risk?

Income may dominate the distance calculation unless features are scaled.

A delivery-time value is empty. What data-quality problem is this?

Missing value (completeness problem).

A distance is recorded as −3 km. What problem is this?

An invalid value (validity problem).

A numerical column contains 'forty'. What problem is this?

An inconsistent data type or format.

The same order appears twice. What problem is this?

Duplicate records (uniqueness problem).

A column contains 'yes', 'Yes', and 'YES'. What problem is this?

Inconsistent formatting or capitalization.

A model learns from emails labeled spam or not spam. Learning type?

Supervised learning, because the correct labels are provided.

An algorithm groups customers without labels. Learning type?

Unsupervised learning, specifically clustering.

Training accuracy is 99%, test accuracy is 60%. What is likely wrong?

Overfitting: the model learned training-specific patterns.

Name two general ways to reduce overfitting.

Use a simpler model and regularization.

A decision tree overfits. What setting could you change?

Reduce its maximum depth or prune the tree.

Why keep validation and test sets separate?

Validation tunes the model; the untouched test set evaluates it fairly.

What is feature selection?

Choosing useful features from the existing dataset.

What is feature extraction?

Transforming data into a new set of useful features.

Feature selection.

Keeping mileage and engine size from a car dataset is what process?

Using PCA to transform several car measurements is what process?

Feature extraction.