SEIS 763 Machine Learning - Vocabulary Flashcards (Fall 2025)

Course Logistics and Orientation

• Course: SEIS 763 Machine Learning (Fall 2025)

• Welcome message: Lecture will be recorded

• Canvas link: https://stthomas.instructure.com/courses/81396

• Start Here: Syllabus, Modules, Welcome content

Course Goals

• Build foundational Machine Learning concepts and algorithms

• Understand basic Math principles that guide Machine Learning

• Use off-the-shelf Python ML Libraries

• Learn standard Training Pipelines

  • Data preparation

  • Model selection

  • Model training and evaluation

Course Boundaries (What this course won’t do)

• Not a Python tutorial

• Not intended to make you an ML expert (yet)

• Not focused on advanced concepts (deep learning, reinforcement learning, computer vision, natural language processing)

• Not about model deployment

Prerequisites

• SEIS 631: Data Preparation and Analysis

• SEIS 632: Data Analytics and Visualization (may be taken concurrently)

• If you’re not comfortable with Python, go over tutorials

Start Here / Welcome Content (Overview)

• Machine Learning builds computational systems that learn from and adapt to data

• ML is essential in IT today and underpins applications across engineering, medicine, finance, and commerce

• Course covers widely used supervised and unsupervised ML algorithms in technical depth

• Emphasis on theoretical underpinnings and hands-on implementation

• Students will learn to evaluate effectiveness and avoid common pitfalls when applying ML

Topics and Practice: The Coding Pyramid

• The Coding Pyramid concept:

  • No Code

  • Open Source

  • From Scratch

• Indicates progression from high-level tools to deep, hands-on implementation

AI, ML, and Deep Learning: Quick Clarifications

• Artificial Intelligence (AI): broad field of making machines exhibit intelligent behavior

• Machine Learning (ML): subset of AI focusing on learning from data to make predictions

• Deep Learning (DL): subset of ML using deep neural networks

• The slide emphasizes this hierarchy and practical distinctions

A Brief History of Artificial Intelligence (Timeline Highlights)

• 1943: The Turing Test concept introduced via natural language interaction framework

• 1950: Turing’s Computing Machinery and Intelligence (proposes the Turing Test as a measure of machine intelligence)

• 1951–1956: Dartmouth Conference marks birth of AI as a field; early milestones in AI

• 1957: Rosenblatt’s Perceptron—the first artificial neural network capable of learning

• 1965: Weizenbaum develops ELIZA, an early natural language processing program that simulates conversation

• 1967: Newell & Simon develop the General Problem Solver (GPS)

• 1974: The first AI winter begins due to unmet expectations and limited progress

• 1980: Expert systems gain popularity for financial forecasting and medical diagnoses

• 1986: Learning representations by back-propagating errors enables training of deeper networks (Hinton, Rumelhart, Williams)

• 1997: IBM Deep Blue defeats Garry Kasparov in chess

• 1999–2010s: NLP and DL advances accelerate; more milestones listed below

• 2011: IBM’s Watson defeats Jeopardy! champions

• 2012: DeepMind’s work gains prominence in deep learning applications

• 2014: DeepFace (Facebook) achieves near-human facial recognition accuracy

• 2015: AlphaGo (DeepMind) defeats top Go player Lee Sedol

• 2017: AlphaZero defeats best chess and shogi engines

• 2020: OpenAI GPT-3 marks a major breakthrough in NLP

• 2021: DeepMind’s AlphaFold2 solves protein folding problem

• 2022: Controversies around LLMs (e.g., LaMDA) and its perceived sentience

• 2023: Legal actions against AI-art tools over copyright concerns (Stability AI, DeviantArt, Midjourney) for remixing artworks

• Additional notes include DeepMind’s acquisition by Google (~2014) for ~$500M, AlphaZero’s successive victories, etc.

• The timeline highlights ethical, legal, and societal implications associated with AI and ML (e.g., copyright, accountability, transparency)

The Human-Reasoning Bridge: Theory vs Practice

• The course emphasizes walking the tight rope between theory and practice

• The idea is to understand theory deeply while gaining hands-on experience with real data

The Coding Pyramid (Revisited)

• From scratch to open source to no-code tools; progressively build intuition and capability

What is Machine Learning? (Core Idea)

• Given some clues (features), guess the answer (prediction)

• Features are the inputs that describe the data

• The learning process optimizes how to map features to outputs

• The prediction is denoted by ŷ (hat y)

• Fundamental question: how to choose a function (model) that generalizes well to unseen data

What is the Machine Learning Problem? (Formal View)

• Given clues/features, find their importance or weights that best predict the answer

• In practice: learn weights W such that the model output matches the true labels

• Intuition: adjust weights to improve predictions on training data and generalize to new data

The 20 Questions Analogy

• ML can be thought of as iteratively asking questions to home in on the right answer by adjusting internal parameters (weights) based on feedback (loss)

Components of a Machine Learning System

• Features

• Label (Output / Target)

• Loss (Cost) Function

• Optimizer (Algorithm to adjust weights)

• Model (The function that maps features to predictions)

Features, Labels, and the Learning Objective

• Features: x (input vector)

• Label: y (true output)

• Prediction: ŷ (predicted output)

• Loss measures how far ŷ is from y

• Objective: learn weights W (and bias b) to minimize the loss

Features: A Simple Intuition with Numbers

• What number am I thinking? example using digits 0–9 to illustrate features and classification

• Visualization exercises use numbers with specific properties (start-end, curved edges, self-intersections) to motivate feature selection and decision boundaries

• This kind of exercise motivates the idea that features describe data points and that a learning system will try to separate or predict based on those features

The Linear Model: Form and Intuition

• Model form (regression):

  • Pointwise form: yi = W^T xi + b

  • Matrix form: ŷ = X W + b

• Notation:

  • n: total samples

  • d: number of features (dimensions)

  • X ∈ R^{n×d} (rows are samples, columns are features)

  • W ∈ R^{d×1}

  • b ∈ R (bias term)

• Alternative perspective: sometimes written as y = XW + b with y ∈ R^{n×1}

• Geometric interpretation: W contains discriminative power for each feature; larger magnitude means the feature has more influence on the prediction

Two Common Linear Forms (Equivalent Views)

• View 1 (sample-wise): yi = W^T xi + b

• View 2 (dataset-wide): ŷ = X W + b

• The goal: find W and b that minimize the chosen loss on the training data

The Learning Question: How Do You Learn W?

• You define a loss function that measures prediction error

• You then optimize W (and b) to minimize the loss

• The idea is to adjust W in the direction that reduces the loss (gradient-based optimization in many cases)

• In class, a simple illustrative objective is shown as finding the slope W that minimizes the Mean Squared Error (MSE)

Loss (Cost) Function: Why It Matters

• Loss measures the discrepancy between true labels y and predictions ŷ

• Common choice for regression: Mean Squared Error (MSE)

• Formula:
  J(W) = rac{1}{n}
  \sum{i=1}^{n} (\hat{y}i - y_i)^2

• Aim: find W (and b) that minimize J(W)

Conceptual Visual: Gradient Descent (Intuition)

• Loss landscape: as you adjust W, the loss value changes

• The optimizer moves in the direction of decreasing loss (toward a minimum)

• Special case shown: the line slope W that minimizes J, yielding perfect or near-perfect predictions on training data

A Simple Linear Regression Example (Key Takeaways)

• Model: ŷ = W x + b

• If W = 1 and b = 0, then ŷ = x (identity mapping)

• If W ≠ 1 or b ≠ 0, predictions scale/shift relative to x

• The optimal W* and b* minimize J(W, b)

• Empirical demonstration: a table shows different W values with corresponding loss, highlighting that W* produces the smallest loss

• Notation for the learned model: y = W^T x + b

Regression vs Classification: Conceptual Difference (Brief Mention)

• Regression: predict a continuous value (e.g., height, weight, flight duration)

• Classification: predict a discrete label (e.g., male/female; digit类别)

• The same linear model concepts apply with different loss functions (e.g., MSE for regression, cross-entropy for classification)

The Standard Data Pipeline (High-Level)

• Practical sequence:

  • Data collection

  • Data labeling

  • Data preparation

  • Model selection

  • Model training

  • Model evaluation

  • Model deployment

• Reference slide emphasizes a standard, end-to-end workflow

Practical Demo and Assignments (What’s Coming)

• Python Demo and Assignment 1 components:

  • conda setup

  • Jupyter Lab

  • Python basics

  • Submission guide

• Assignment 1 details:

  • Due 09/16/2025 at 5:00 PM

  • Tests basic Python skills

  • Loading multiple files

  • Basic data operations using pandas

  • Plotting using Matplotlib

  • Data source: 20 DL160 flights (MSP-AMS)

  • Data fields: Timestamp, Position, Altitude, Speed, Direction

  • Deliverables: plot and generate insights

• If you struggle, review Python resources and consider taking the class next semester

Reference Plots for Flight Data (Q5 & Q6 Examples)

• Q5 Reference Plot: Flight Duration (HH:MM) vs. Departure Date (2023-10-)

  • Example labels shown: Flight Duration (HH:MM) and Departure Date

• Q6 Reference Plot: Time (HH:MM) breakdown by phase

  • Phases: On-ground before takeoff, In-air time, On-ground after landing

• These plots illustrate typical ML/data visualization tasks (time-series, phase breakdowns, distributions)

Formulating a Machine Learning Problem (Detailed View)

• Input values/features: 𝑥

• Description of the number 8 as an example feature description

• Label/Output: 𝑦

• Objective: Learn weights W such that 𝑥 W = 𝑦

• For 1D inputs, this reduces to a simple linear relation: y = W x + b

Linear Model Details: Notation and Dimensions

• Model: y = W x + b

• General form for multiple samples:

  • y ∈ R^{n×1} (n samples)

  • X ∈ R^{n×d} (features)

  • W ∈ R^{d×1}

  • b ∈ R (bias, broadcast to all samples)

• Transposed form (for other conventions):

  • y^T = W^T X^T + b^T

• The essence: labels are predicted via a weighted combination of features plus a bias term

Label, Feature, Loss, Optimizer, and Model (Summarized)

• Label: y (true value)

• Features: x (input vector)

• Loss: measures prediction error between y and ŷ

• Optimizer: algorithm that updates weights to reduce loss

• Model: the functional form that maps features to predictions

A Simple 1-D Example: Feature-Based Classification/Regression Intuition

• Features can be scalars or vectors; weights combine them into a prediction

• The loss function guides how to adjust the weights to better fit the data

Height/Weight Distribution by Sex (Example Visualization)

• A sample scatter-like concept: Height (in) vs Weight (lbs), separated by Sex (Male, Female)

• Demonstrates how linear relationships can differ across groups and how features might separate classes or predict a continuous target differently by group

Transpose and Shape Notes (Coding vs Theory)

• You can express the same linear model in multiple equivalent ways depending on the matrix orientation

• Two common forms shown:

  • ŷ = X W + b (n×d times d×1 = n×1)

  • yi = W^T xi + b (individual sample form)

• Dimensional consistency: n = number of samples; d = number of features

Practical Visualization and Interpretation (Homework of the Day)

• The course uses simple numeric demonstrations (e.g., digits 0–9 with features such as start/end, curves, intersections) to illustrate feature extraction and decision boundaries

• These exercises emphasize understanding what features convey about the target and how a model uses them

The Data Pipeline in Practice: Summary

• Data collection → Data labeling → Data preparation → Model selection → Model training → Model evaluation → Deployment

• Each step is essential for building reliable ML systems

Assignments and Resources Summary

• Assignment 1:

  • Python skills test; file loading; pandas operations; plotting with Matplotlib

  • Data source: 20 DL160 flights (MSP-AMS) with fields: Timestamp, Position, Altitude, Speed, Direction

  • Emphasis on practicing data handling and visualization to extract insights

• Resources: Python tutorials and class tutorials; if needed, consider taking the course next semester

Final Quick Reference: Core Equations and Concepts

• Linear model (regression):

  • Pointwise form: yi = W^T xi + b

  • Dataset form: ŷ = X W + b, with X ∈ R^{n×d}, W ∈ R^{d×1}, b ∈ R

• Loss (Mean Squared Error):

  • J(W) = \frac{1}{n} \sum{i=1}^{n} (\hat{y}i - y_i)^2

• Objective: find W and b that minimize J(W) (and generalize to unseen data)

• Conceptual interpretation: Weights capture the discriminative power of features

• Data pipeline components: Features, Label, Loss Function, Optimizer, Model

• Model understanding: Your brain builds a model; in ML, we learn a model by optimization

• Ethical/Practical implications highlighted in course timeline: copyright concerns in AI-generated art, governance of AI systems, and accountability for outputs

Quick Index of Key Terms from Slides

• Artificial Intelligence (AI)

• Machine Learning (ML)

• Deep Learning (DL)

• Perceptron, ELIZA, GPS, Deep Blue, Watson, DeepFace, AlphaGo, AlphaZero, GPT-3, AlphaFold2

• Turing Test, Dartmouth Conference, ELIZA, Backpropagation, Neural Networks

• Loss Function, MSE, Optimizer, Model, Features, Label

• Data Pipeline, Convolution, Go/Chess/Go-like strategic ML milestones, Ethical implications (copyright lawsuits)