Lecture 31 Making Rational Decisions

Flashcards on Decision Theory, Utility Functions, and Markov Decision Processes.

Decision Theory

The study of how to make rational decisions, often modeled using utility functions.

Utility Function

A function that models rational behavior by assigning values to different outcomes.

Markov Decision Process (MDP)

A framework for making sequences of decisions in an uncertain environment, using probabilities to express uncertainty.

Reinforcement Learning

An area of Machine Learning closely related to solving Markov decision processes, but without explicit transition probabilities or reward functions.

Transition Model

Probabilities associated with transitioning between states in a Markov Decision Process.

Reward Function

A function that defines the rewards or penalties associated with different states or actions in a Markov Decision Process.

Von Neumann Architecture

The basic design for modern computers, co-developed by John von Neumann, who also contributed to utility theory.

Neumann-Morgenstern Utility Theorem

A theoretical justification for using utility functions to model rational behavior.

Lottery

A representation of actions in an uncertain environment where outcomes have associated probabilities.

Maximum Expected Utility Principle

The principle that the most rational choice is the action with the highest expected utility, considering probabilities and utility values.

Normalized Lottery

A method used to elicit utility functions from humans by comparing certain outcomes to a lottery with best and worst possible outcomes.

Axioms of Utility Theory

Rules that define rational behavior, used as the foundation for utility theory.

Transitivity

If A is preferred to B, and B is preferred to C, then A is preferred to C. It is a common sense rule regarding rational behavior of an agent.

Expected Utility

The sum of the utilities of each possible outcome of a lottery, weighted by their respective probabilities.

Linear Transformation of Utility Function

Multiplying utility function by a constant and adding a constant creating new utility function.

Conditional Probability

The probability of an event occurring given that another event has already occurred.

Elicit Utility Function

To get a utility function from people or expert.

Standard Lottery

Lottery where we have the best possible state as one outcome and the worst possible state as the other outcome.

Expected Monetary Value (EMV)

The expected value of a lottery, calculated by multiplying the value of each outcome by its probability and summing the results.