CP468 Midterm

Four Ways to Define AI

Acting humanly: Turing Test
Thinking humanly: cognitive science
Thinking rationally: laws of thought
Acting rationally: (AI) rational = doing the right thing

Rule-based systems

Explicit IF-THEN logic

ex: If email contains “lottery” → spam

Learning-based systems

Patterns extracted automatically from data

data → model → prediction

ex: traditional LLM

Pros and Cons of Learning-based vs rules based

Learning-based:
Pro: Flexible and powerful
Cons: Less transparent, hard to explain why it made decision

Rules-based:
Pro: Fast, transparent, predictable
Con: breaks down in reality, no one can write enough rules

3-Types of Learning

Supervised
Unsupervised
Reinforcement

Supervised Learning

Learns from labelled data. Inputs paired with correct outputs. The model maps one to the other.

ex: Search ranking, Spam vs not-spam, fraud detection

Unsupervised Learning

Learns without labels

ex: Clustering

Reinforcement Learning

Takes actions in environment, receive rewards or penalties, improve over time

Discriminative vs Generative

Discriminative: Given this input, which class does it belong to? (ex: Logistic Regression)
Generative: Given this distribution, produce new examples that look like real data (ChatGPT)

AI Problem in 5 Parts SATGC

State: What does the world look like right now? (Position)
Actions: What can’t the agent do? (Move, Buy)
Transition: What happens after an action? (New state + action)
Goal: What are we trying to achieve? (Checkmate)
Cost: What do we want to minimize? (Time, money)

Data is the ___ of AI

fuel

Data ____ matters more than ________

Data quality matters more than model complexity

More data =

Generally improves performance up to a point

Biased Data =

Model learns whatever is in the training set

Learning vs Memorization

Good AI: generalizes, learns patterns
Bad AI: memorizes training data, fails on new examples

Evaluation for AI

Classification, Regression, Reinforcement Learning

Classification(predict category): Accuracy
Regression(predict number): Error
Reinforcement Learning(maximize goal): Reward

Limitations/Risks of AI (BPHO)

Bias: model learns inequities baked into training data
Privacy: Training data can leak into output
Hallucinations: Confidently incorrect outputs
Over-reliance: trusting AI more than it deserves

Agent Definition

An agent perceives it’s environment through sensors and acts on it through actuators

Agents include…

Humans
Robots
Softbots (API calls in, API calls out)
Thermostats (Temp sensor, turn on heater)

Agent Function vs Agent Program

Agent Function(abstract): Maps a percept history to an action
Agent Program(concrete): Code that runs on physical architecture to produce agent behaviour

Percepts

pieces of information an AI agent receives from its environment through sensors

ex: [location, contents] → [A, Dirty] (for vacuum)

Rationality

Rational agent picks option that is expected to maximize it’s performance

Not: Omniscience, clairvoyance, success

Is: Exploration, learning, autonomy

PEAS: how to specify task enviorment

P: performance (how do we score success?)
E: environment (what world does agent live in?)
A: actuators (What can the agent do to/in world?)
S: sensors (What can the agent perceive?)

PEAS for Shopping Agent

Peformance: Price, Quality, Good for User?
Environment: Current and future websites, vendors, the user
Actuators: render results to user, follow URLs, place orders
Sensors: HTML pages, user clicks and queries

Todays LLM agents are a type of…

Softbot

Softbot

An AI genet that mimics human tasks, acts on users behalf, or serves as digital bridge between different computer systems

Automation tools and industrial robotics software

Fully vs Partially Observable (7-Dimensions Environment)

Can the agent’s sensors see the complete state at each step?

Deterministic vs Stochastic (7-Dimensions Environment)

Is the next state fully determined by the current state and action?

Episodic vs Sequential (7-Dimensions Environment)

Do current decisions affect later ones?

Static vs Dynamic (7-Dimensions Environment)

Does the environment change while the agent is deliberating?

Discrete vs Continuous (7-Dimensions Environment)

Are time, percepts, and actions countable or smoothly varying

Single-agent vs multi-agent (7-Dimensions Environment)

Are there other agents whose behaviour matters? Cooperative or adversarial?

Known vs Unknown (7-Dimensions Environment)

Does the agent know the rules of the environment in advance

Environment type drives architecture choice. Use the ______ agent the world allows.

Environment type drives architecture choice. Use the simplest agent the world allows

5 Agent Architectures, in order (SMGUL) (STAR)

1. Simple Reflex Agent (acts on percept only)

2. Model-based reflex agent (maintains internal state of the world)

3. Goal-based agent (reasons about how to reach a goal)

4. Utility-based agent (Maximizes a continuous utility, not just goals)

5. Learning Agent (Any of the above + improves from experience)

Simple-Reflex Agent & Pros Cons

Match the current percept against a set of condition-action rules. Whichever rule fires first wins.

Pros: Cheap, fast, predictable
Cons: Has no memory, may loop forever

Model-based reflex agent & Two Ingredients

Keeps an internal model of the world to handle parts the sensors don’t currently see.

Two Ingredients:
How the world evolves (Cars move when I look away)
What my actions do (If I turn wheel left, my heading changes)

Goal-based agent

Considers possible futures “What will happen if I do A?” then chooses the action whose predicted outcome is closest to the goal

Utility-based agent

Maps a state to a real number, “how happy” the agent would be there. Choose the action whose expected utility is highest

Goals-based vs Utility-based agent

Goals → binary (reached or not)
Utility → a continuous degree of preference

Learning Agent and 4 components (PCLP)

Performance element → selects actions
Critic → compares behaviour to a fixed performance standard
Learning element → Updates the performance element from the critics feedback
Problem generator → suggests exploratory actions to discover new things

LLM Agent Anatomy (STAR)

Agent = LLM + Memory + Planning + Tool Use

Short-Term vs Long-Term memory LLM

Short-Term: Context Window
Long-Term: Vector DB, Files, Summaries

Agent =

Agent = percieve → think → act

Problem Solving Agent Implementation:
On holiday in Romania; currently in Arad. Flight leaves
tomorrow from Bucharest

Goal: be in Bucharest
Problem:
- States: various cities
- Actions: drive between cities
Solution: sequence of cities

Problem-solving process

1. Goal formula

2. Problem: state and actions

3. Search: simulate actions until reaches goal (Solution)

4. Execution

State

description of the world at a particular moment in time

Action

Something the agent can do that causes a transition from one state to another

Transition Model

Agents internal representation of how the world works, how actions change states

(Domain) Model

Complete abstract description of the problem world: states, actions, transitions, goal, coast

Optimal Solution

Lowest path cost among all solutions

Domain Model Abstraction

Ignore irrelevant states. Replacing real-world complexity with simpler representations

ex: Map of cities → cities on graph with distances between them

Vacuum World (States, Actions, Goal Test, Path Cost)

states: integer dirt and robot locations
actions: left, right, suck, NoOp
goal test: no dirt
Path cost: 1 per action

Tree Search Algo

States vs Nodes

State: representation of a physical configuration
Node: data structure constituting part of a search tree (parent, children, depth, path cost g(x))

Strategies are evaluated by the following dimensions (CTSO)

Completeness: Does it always find a solution if one exists
Time complexitity
Space complexity
Optimality