L4 - Agents and Environments

True or False. Agent = Architecture + Program

True

Fill in the gap. Agents should be….

Rational

What does the behaviour of the agent depend on?

Environment and the defined goals

What can agents include?

Humans, robots, softbots, thermostats, etc.

True or False. The agent function maps from percept histories to actions

True

True or False. The agent program runs on physical architecture to produce the agent function

True

What does an agent use to perceive it’s environment?

Sensors and actuators

Diagram of classical agent

Diagram of a modern LLM agent

What are the characteristics of an intelligent agent?

Autonomy

Reactivity

Proactivity

Social ability

What are some examples of an intelligent agent?

Robots, software agents, virtual assistants

What would be the percepts of a robot vacuum-cleaner?

Locations and contents

E.g., [A, Dirty, B, Clean]

What would be the actions of a vacuum-cleaner agent?

Left, Right, Inhale

Diagram of an example of percept sequences and actions for a vacuum-cleaner agent

Diagram of an example of a function for a vacuum-cleaner agent

What is the first dependent of rationality for an agent?

The performance measure defines the criteria of success

What is the second dependent of rationality for an agent?

The agent’s prior knowledge of the environment

What is the third dependent of rationality for an agent?

The actions the agent can perform

What is the fourth dependent of rationality for an agent?

The agent’s percept subsequence to date

True or False. A rational agent chooses whichever action maximises the expected value of the performance measure given the percept sequence to date.

True

Diagram of discussion of the attributes of rationality for an agent

What four factors are used to specify the task environment to design a rational agent?

Performance measure

Environment

Actuators

Sensors

What is meant by performance measure?

How the agent’s success is measured

What is meant by environment?

The surroundings in which the agent operates

What is meant by actuators?

Tools agent uses to take action

What is meant by sensors?

Tools agent uses to perceive its environment

Diagram of a specified task environment for an automated taxi

Diagram of a specified task environment for a internet shopping agent

Diagram of a specified task environment for ChatGPT assistant agent

Diagram of the profound shift from chatbots to copilots to agents

True or False. The LLM is the agent.

False. The LLM is only the brain, it needs a harness to be useful

What is meant by a harness in modern AI agents?

A system that supports and controls an AI agent, so it can perform tasks

True or False. There are six factors included in a harness.

True.

Instructions

Tools & APIs

Memory & context

Execution runtime

Observations

Controls & approvals

What are instructions to an LLM?

The rules the AI operates by

What are tools & APIs to an LLM?

What the AI can call

What is memory & context to an LLM?

What the AI knows & remembers

What is execution runtime to an LLM?

Where the AI’s work is performed

What are observations to an LLM?

Feedback the AI receives about its actions

What are controls & approvals to an LLM?

Permissions determining what the AI is allowed to do

Which three factors make up an AI system?

Model, harness and environment

Diagram of all components in an AI system

What does observability describe in an environment?

Whether the agent can see all relevant aspects

What is an example of a fully observable environment?

Chess. Whole board is visible

What is an example of a partially observable environment?

Poker

What is the difference between a deterministic and stochastic environment?

Deterministic - Current state and action determines next state

Stochastic - Next state is not completely predictable

What is the difference between a episodic and sequential environment?

Episodic - Agent experience divided into independent episodes

Sequential - Decisions depend on each other

What is an example of a episodic environment?

A quiz game

What is an example of a sequential environment?

Football

What is the difference between a static and dynamic environment?

Static - Environment doesn’t change, while agent deliberates

Dynamic - Environment can change even, if agent does nothing

What is an example of a static environment?

Chess

What is an example of a dynamic environment?

Self-driving cars

What is the difference between a discrete and continuous environment?

Discrete - Finite number of actions and states

Continuous - Range of possible states and actions is infinite

What is an example of a discrete environment?

Tic-tac-toe

What is an example of a continuous environment?

Steering a car

What is the difference between a single-agent and multi-agent environment?

Single-agent - Only one agent acts in the environment

Multi-agent - Multiple agents interact

What is an example of a single-agent environment?

A sudoku solver

What is an example of a multi-agent environment?

A stock trading system

Diagram of games and their environment types

True or False. The real world is fully observable, deterministic, episodic, static, discrete and single-agent.

False. The real world is the complete opposite of all those types.

Table of a modern agentic coding assistant’s environment types

How does a simple-reflex agent work?

Uses sensors to observe the current environment

Follows condition-action rules to choose an action

True or False. A simple-reflex agent remembers past situations

False. Its actions are based off sensing the current environment

Diagram of a simple reflex agent

True or False. A goal-based agent uses only its initial environment to decide which action to take, not considering its goals.

False. Uses the current environment and its goals to decide which action will help achieve the desired result

Diagram of a goal-based agent

Which four factors is a problem defined by?

Initial state

Successor function

Goal test

Path cost

True or False. A solution is a sequence of actions from the initial to a goal state

True

Example of a goal-based agent travelling to Romania

Example problem formulation for travelling to Romania

What is the first step a utility-based agent follows?

Perceives the environment

What is the second step a utility-based agent follows?

Predicts outcomes of possible actions

What is the third step a utility-based agent follows?

Evaluates outcomes using a utility function

What is the fourth step a utility-based agent follows?

Chooses best outcome

Diagram of a utility-based agent

What are some examples of utility-based agents?

Autonomous vehicles

Financial trading systems

Recommendation systems

Healthcare management

What is the first step a learning agent follows?

Takes feedback from a critic to learn from its performance

What is the second step a learning agent follows?

Updates its knowledge to improve its performance

What is the third step a learning agent follows?

Uses a problem generator to explore new actions and improve future decisions

What are some examples of learning agents?

Game Playing AI (AlphaGo, Chess AI)

Recommendation Systems (Amazon)

How does a deterministic, fully observable (single-state) problem work?

Agent knows which state it will be in

Solution is a sequence

How does a non-observable (sensor-less) problem work?

Agent may have no idea where it is

If solution, it’s a sequence

How does a non-deterministic and/or partially observable (contingency) problem work?

Percepts provide new info about current state

Solution is contingent plan or policy

How does a unknown state space (exploration) problem work?

Agent must explore environment to discover the states and how they connect

Current research and future directions

What are some examples of multi-agent systems?

Traffic management

Smart grids

Robotics

Gaming

What are some challenges in developing multi-agent systems?

Communication and coordination

Decision-making

Learning

What are some ethical and social implications?

Algorithm bias and fairness

Transpareny

Privacy and security

Legal and accountability

Workplace displacement