AI 5.2

What is a random variable?

A random variable is a variable whose possible values are numerical outcomes of a random phenomenon.

What is probability?

Probability is a number between 0 and 1 that measures how likely an event is to occur.

What is a joint probability distribution?

A joint probability distribution gives the probability of every combination of values for a set of random variables.

Define conditional probability.

Conditional probability measures the probability of an event given that another event has occurred. It is written as P(A|B).

State Bayes' Rule in plain text.

P(A|B) = P(B|A) * P(A) / P(B)

What is marginalization?

Marginalization is the process of summing (or integrating) over unwanted variables to obtain the probability distribution of a subset.

What is a Bayesian network?

A Bayesian network is a graphical model that represents probabilistic relationships among a set of variables using a directed acyclic graph and conditional probability tables.

What are the two components of a Bayesian network?

A directed acyclic graph (DAG) and conditional probability tables (CPTs) attached to each node.

How is the joint distribution represented in a Bayesian network?

P(X1, X2, …, Xn) = product over i of P(Xi | Parents(Xi))

In the simple Rain/Wet Grass example, what does P(R=1)=0.4 mean?

The prior probability that it rains on any given day is 40%.

In the Rain/Wet Grass example, what does P(W=1|R=1)=0.9 mean?

The probability that the grass gets wet given that it is raining is 90%.

In the Rain/Wet Grass example, what does P(W=1|R=0)=0.2 mean?

The probability that the grass gets wet given that it is not raining (e.g., sprinkler on) is 20%.

What is inference in a Bayesian network?

Inference is the process of answering questions about the underlying probability distribution given some observations.

What is a diagnosis question in inference?

A diagnosis question asks for the probability of a cause given an observed effect. Example: P(Rain | Wet Grass).

What is a prediction question in inference?

A prediction question asks for the probability of an effect given a cause. Example: P(Wet Grass | Rain).

Give one application of inference in Bayesian networks.

Classification: finding the most probable class label given data.

Give another application of inference in Bayesian networks.

Decision Making: combining probabilities with utilities to choose an action.

In the simple Rain/Wet Grass example, what is P(W=1)?

P(W=1) is the marginal probability that the grass is wet, calculated as P(W=1|R=1)P(R=1) + P(W=1|R=0)P(R=0).

In the simple Rain/Wet Grass example, what was the calculated value of P(W=1)?

0.48

In the simple Rain/Wet Grass example, what is the diagnosis result P(R=1|W=1)?

0.75

Interpret the result P(R=1|W=1)=0.75 in the simple example.

Knowing that the grass is wet increases the probability that it rained from the prior 0.4 to 0.75.

What are evidence variables?

Evidence variables are the variables whose values we know (observed).

What are query variables?

Query variables are the variables whose probability distribution we want to find.

What are non-evidence (hidden) variables?

Hidden variables are all other variables that are neither observed nor queried but must be accounted for in the joint distribution.

What is an unconditional probability query?

An unconditional probability query asks for the probability of a given value assignment for a subset of variables, e.g., P(W=1).

What is a conditional probability query?

A conditional probability query asks for the probability of query variables X given evidence about variables E, e.g., P(W|C=1).

What is Maximum a Posteriori (MAP) inference?

MAP inference finds the most likely assignment of values to the query variables X given evidence E = e.

Write the formula for MAP inference in plain text.

MAP(X | E=e) = argmax_x P(X=x | E=e)

What does the symbol α represent in exact inference?

α is a normalizing constant equal to 1 / P(E), which ensures the posterior distribution sums to 1.

Write the general formula for exact inference in plain text.

P(X | E) = α * sum over Y of P(X, E, Y)

In the full Wet Grass example, what are the four variables?

Cloudy (C), Sprinkler (S), Rain (R), Wet Grass (W).

Write the factorization for the full Wet Grass example in plain text.

P(C,S,R,W) = P(C) * P(S|C) * P(R|C) * P(W|S,R)

In the full Wet Grass example, if C=1, what is P(S=1|C=1)?

0.1

In the full Wet Grass example, if C=1, what is P(S=0|C=1)?

0.9

In the full Wet Grass example, if C=1, what is P(R=1|C=1)?

0.8

In the full Wet Grass example, if C=1, what is P(R=0|C=1)?

0.2

In the full Wet Grass example, what is P(W=1 | S=0, R=0)?

0.0

In the full Wet Grass example, what is P(W=1 | S=0, R=1)?

0.9

In the full Wet Grass example, what is P(W=1 | S=1, R=0)?

0.9

In the full Wet Grass example, what is P(W=1 | S=1, R=1)?

0.99

In the full Wet Grass example, what is the calculated value of P(W=1 | C=1)?

0.7452

What is the purpose of marginalization?

To eliminate variables from a joint distribution to obtain the distribution of a subset.

For discrete random variables, how is marginalization performed?

By summing over the unwanted variable.

For continuous random variables, how is marginalization performed?

By integrating over the unwanted variable.

What does the joint probability table on slide 15 illustrate?

It illustrates how to obtain marginal probabilities by summing rows or columns.

Name one advanced method for exact inference.

Variable Elimination.

Name another advanced method for exact inference.

Clustering Algorithm (Junction Tree Algorithm).

Name a third advanced method for exact inference.

Sum-Product Algorithm (Belief Propagation).

What is the key idea of Variable Elimination?

It systematically sums out variables one by one, reusing intermediate results to reduce computation.

What is the key idea of the Clustering Algorithm?

It groups variables into clusters to turn the network into a tree, then performs inference efficiently.

What is the key idea of the Sum-Product Algorithm?

It propagates messages along the graph to compute marginal probabilities, especially effective in tree-structured networks.

Write the formula for Bayes' Rule as in the summary.

P(A | B) = P(B | A) * P(A) / P(B)

Write the formula for marginalization of discrete variables as in the summary.

P(X=x) = sum over y of P(X=x, Y=y)

Write the formula for the joint distribution in a Bayesian network as in the summary.

P(X1, X2, …, Xn) = product over i of P(Xi | Parents(Xi))

Write the formula for posterior probability (exact inference) as in the summary.

P(X | E) = α * sum over Y of P(X, E, Y) where α = 1 / P(E)

Write the formula for MAP inference as in the summary.

MAP(X | E=e) = argmax_x P(X=x | E=e)

Write the formula for P(W=1 | C=1) from the Wet Grass example as in the summary.

P(W=1 | C=1) = sum over S,R of P(S | C=1) * P(R | C=1) * P(W=1 | S, R)