Artificial Intelligence - Bayesian Networks and Probability Basics

Flashcards testing core concepts of probability basics, probabilistic inference, Bayes' rule, and Bayesian networks.

How is expected utility maximized when making decisions under uncertainty?

Decision making combines probability and utility to maximize expected utility using the formula $a^* = \text{argmax}_a \sum_s P(s \mid a) U(s)$.

What two conditions must a discrete probability distribution $P(\omega)$ satisfy over a set of outcomes $\Omega$?

1) $0 \le P(\omega)$ for all outcomes $\omega$; 2) $\sum_{\omega \in \Omega} P(\omega) = 1$.

How is the probability of an event $A$ calculated from individual outcome probabilities?

The probability of an event $A$ is the sum of probabilities over its outcomes: $P(A) = \sum_{\omega \in A} P(\omega)$.

What is the formal definition of a random variable?

A random variable is a deterministic function of an outcome $\omega$.

Why can marginal distributions be calculated from a joint distribution, but joint distributions cannot be calculated from marginal distributions alone?

Because marginal probabilities do not contain information about how variables are related to each other, whereas joint distributions capture all dependencies between variables.

What formula is used to marginalize (sum out) variable $Y$ from a joint distribution to obtain $P(X=x)$?

$$P(X=x) = \sum_y P(X=x, Y=y)$$

What is the size of a full joint distribution table for $n$ variables each having domain size $d$?

The size of the joint distribution is $d^n$.

What mathematical equation defines the independence of two random variables $X$ and $Y$?

$\forall x,y \, P(x, y) = P(x) P(y)$, or equivalently $P(x \mid y) = P(x)$.

What is the formula for conditional probability $P(a \mid b)$?

$$P(a \mid b) = \frac{P(a, b)}{P(b)}$$

How is the normalization factor $\alpha$ defined when normalizing a probability distribution?

$\alpha = \frac{1}{\sum \text{entries}}$ where the sum is taken over all unnormalized entries in the distribution.

What is the Product Rule formula for probability?

$$P(a, b) = P(a \mid b) P(b)$$

What is the general expression for the Chain Rule of probability for $n$ variables?

$$P(x_1, x_2, \dots, x_n) = \prod_i P(x_i \mid x_1, \dots, x_{i-1})$$

What are the three steps in performing probabilistic Inference by Enumeration to calculate $P(Q \mid e)$?

1) Select the entries consistent with the evidence $E$; 2) Sum out hidden variables $H$ to obtain the joint distribution of query and evidence; 3) Normalize the resulting distribution.

What are the time and space complexities of exact inference by enumeration for $n$ variables with domain size $d$?

Both time complexity and space complexity are $O(d^n)$.

What is Bayes' Rule and what are its components called?

$P(a \mid b) = \frac{P(b \mid a) P(a)}{P(b)}$, where $P(a)$ is the prior probability, $P(b)$ is the evidence, $P(b \mid a)$ is the likelihood, and $P(a \mid b)$ is the posterior probability.

What is the definition of conditional independence for variable $X$ given variable $Z$ with respect to variable $Y$?

$\forall x,y,z \, P(x \mid y, z) = P(x \mid z)$ (or equivalently $\forall x,y,z \, P(x, y \mid z) = P(x \mid z) P(y \mid z)$).

What defines a Naïve Bayes model?

A Naïve Bayes model consists of one discrete query variable (class/category) where all evidence variables are conditionally independent given that query variable.

In the provided Bayesian Network diagram, what does the absence of a direct arc between Toothache and Catch signify?

The absence of an arc indicates that Toothache and Catch are conditionally independent given Cavity.

What two components make up a complete Bayesian Network?

1) Topology (a Directed Acyclic Graph representing random variables as nodes and direct influences as arcs); 2) Local Conditional Probabilities (a Conditional Probability Table / CPT for each node).