Chapter 3: Probability

STAT0002 Week 3 — Chapter 3 lecture notes and Tuesday/Thursday Week 3 slides. Probability, types of events, conditional probability, addition and multiplication rules, and independence.

What is a random experiment?

An experiment whose outcome cannot be predicted with certainty.

What is the sample space S?

The set of all possible outcomes of a random experiment.

What is an event E?

A collection of outcomes, possibly just one outcome.

What is the relative frequency (frequentist) definition of probability?

The relative frequency with which the event occurs in the long run; the proportion of times that the event would occur in an infinite number of identical repeated experiments.

How is P(E) expressed using relative frequency?

P(E) is the limiting value of n(E)/n as n → ∞, supposing that this limit exists. n(E) denotes the number of times event E occurs in n experiments.

When choosing an individual at random from a population, what is the probability of a particular property?

The proportion of individuals in the population which have this property. All individuals are equally likely to be chosen.

What does Kerrich’s coin-tossing example illustrate?

Initially the proportion of heads fluctuates greatly but begins to settle down as the number of tosses increases. After 10,000 tosses the relative frequency of Head is 5067/10,000 = 0.5067.

What are the basic properties of probability?

0 ≤ P(E) ≤ 1. If E is impossible then P(E) = 0; if E is certain then P(E) = 1. P(S) = 1 because the outcome must, by definition, be in the sample space.

In what sense are all probabilities conditional probabilities?

A probability is conditioned on the sample space S: defining S rules out anything not in S. For example, instead of P(C) we could write P(C | S). We should always bear in mind the sample space.

What do P(A,B), P(A and B), P(A ∩ B) and P(AB) mean?

The probability that both events A and B occur. P(A,B) = P(B,A).

What does P(A or B), or P(A ∪ B), mean?

The probability that A or B, or both, occur.

When are two events mutually exclusive (or disjoint)?

If they cannot occur together. Then P(A and B) = 0.

What is the addition rule for any two events A and B?

P(A or B) = P(A) + P(B) − P(A and B).

Why does the addition rule subtract P(A and B)?

Adding P(A) and P(B) counts the outcomes in both A and B twice. Subtracting P(A and B) counts them once.

How does the addition rule simplify for mutually exclusive events?

P(A or B) = P(A) + P(B), because P(A and B) = 0.

When are events A and B exhaustive?

If P(A or B) = 1.

What is the complement of an event A, and how is it denoted?

The event that A does not occur. It can be denoted not A, Ā or Aᶜ.

How are A and its complement related?

They are mutually exclusive and exhaustive. P(not A) = 1 − P(A).

What does the | sign mean in P(A | B)?

Conditional on, or given. P(A | B) is the probability of event A given event B.

What is the definition of conditional probability?

If B is an event with P(B) > 0, then for each event A, P(A | B) = P(A,B)/P(B).

How does conditioning on B change the sample space?

The sample space is reduced to outcomes containing the event B. We consider only those occasions on which B occurs.

If P(A) > 0 and P(B) = 0, which version of the multiplication rule can be used?

P(A | B) is undefined because P(B) = 0. Instead use P(A,B) = P(B | A)P(A); here P(B | A) = 0 and P(A,B) = 0.

Do properties of probabilities also hold for conditional probabilities?

Yes. P(A | B) is a probability in a reduced sample space; for example, 0 ≤ P(A | B) ≤ 1.

Do P(A | B) and P(B | A) have the same meaning?

No. P(A | B) is the probability of A given B; P(B | A) is the probability of B given A.

What is an unconditional (or marginal) probability?

The overall probability. In the cot death example, P(C) = 1/1303 for all families, whereas P(C | F₁) = 1/8543. Here C = one cot death and F₁ = an affluent, non-smoking family with a mother aged over 26.

In the Berkeley example, what are P(A | M) and P(A | F)?

With A = accepted, M = male and F = female: P(A | M) = 1198/2691 = 0.445; P(A | F) = 557/1835 = 0.304.

In the Berkeley example, how do P(M,A) and P(A | M) differ?

P(M,A) = 1198/4526 = 0.265 uses all applicants. P(A | M) = 1198/2691 = 0.445 uses only male applicants.

In the Berkeley example, what is P(M or A)?

P(M) + P(A) − P(M,A) = 0.595 + 0.388 − 0.265 = 0.718.

What is the multiplication rule for two events?

If P(B) > 0, P(A,B) = P(A | B)P(B). Also, if P(A) > 0, P(A,B) = P(B | A)P(A).

Does the general multiplication rule require independence?

No. If P(B) > 0, P(A,B) = P(A | B)P(B) is always true. P(A,B) = P(A)P(B) only if A and B are independent.

What is the multiplication rule for three events?

P(A₁,A₂,A₃) = P(A₁)P(A₂ | A₁)P(A₃ | A₁,A₂), if all the conditional probabilities exist. The events can be ordered in other ways.

How does the multiplication rule generalise to n events?

P(A₁,…,Aₙ) = P(A₁)P(A₂ | A₁)P(A₃ | A₁,A₂)⋯P(Aₙ | Aₙ₋₁,…,A₁), provided all the conditional probabilities are defined.

What does P(A | B,C) mean?

The probability that event A occurs given that both events B and C have occurred. We condition on all events placed to the right of the | sign.

How can P(A | B,C) be expressed using joint probabilities?

P(A | B,C) = P(A,B,C)/P(B,C), provided P(B,C) > 0.

When are two events A and B independent?

If P(A,B) = P(A)P(B). Otherwise A and B are dependent events.

What does independence imply about conditional probability?

If P(B) > 0, P(A | B) = P(A): the occurrence of B does not affect the probability of A. Similarly, if P(A) > 0, P(B | A) = P(B).

Is independence symmetric?

Yes. If A is independent of B, then B is independent of A. A ⫫ B means “A and B are independent”.

Can the definition of independence apply to an event with zero probability?

Yes. If P(A) > 0 and P(B) = 0, then A and B are independent because P(A,B) = 0 = P(A)P(B).

Can mutually exclusive events with positive probabilities be independent?

No. For mutually exclusive events P(A,B) = 0, but if P(A) > 0 and P(B) > 0 then P(A)P(B) > 0.

Does P(A | M) > P(A | F) by itself establish gender bias in the Berkeley admissions process?

No. The notes warn that this may be a very misleading conclusion to draw from these data: there is an innocent explanation. Dependence in the overall admissions data does not by itself establish gender bias in the admissions process.

When are three events A, B and C pairwise independent?

When all three hold: P(A,B) = P(A)P(B), P(A,C) = P(A)P(C), and P(B,C) = P(B)P(C).

When are three events A, B and C mutually independent?

They are pairwise independent and P(A,B,C) = P(A)P(B)P(C).

What alternative expressions for P(A | B,C) appear in Tuesday’s poll?

If P(B,C) > 0 and P(A,C) > 0: P(A | B,C) = P(A,B | C)/P(B | C) = P(B | A,C)P(A | C)/P(B | C). Also P(A | B,C) = P(A | C,B): the order of the conditioning events does not matter.

When are A₁,…,Aₙ mutually independent?

For every r ∈ {2,…,n}, and every subset {C₁,…,Cᵣ} of {A₁,…,Aₙ}, P(C₁,…,Cᵣ) = P(C₁)⋯P(Cᵣ).

Is P(A,B,C) = P(A)P(B)P(C) alone sufficient for mutual independence?

No. The pairwise identities must also hold. In the slide’s diagram, P(A) = 0.2, P(B) = 0.4 and P(C) = 0.5, so P(A,B,C) = 0.04 = P(A)P(B)P(C). But P(A,B) = 0.10 ≠ 0.08 = P(A)P(B), so the events are not mutually independent.

What independence example is given by ABO and Rhesus blood types?

ABO and Rhesus blood types are inherited independently: the genetic codes responsible are on different chromosomes. Assuming independence, P(O,Rh+) = P(O)P(Rh+) = 0.44 × 0.83 = 0.3652 (about 0.365).

What was the problem with multiplying 1/8543 by 1/8543 in the Sally Clark example?

It makes an independence assumption for two cot deaths in the same family. The multiplication P(C₁,C₂) = P(C₁)P(C₂) requires independence.

In the Sally Clark example, what probabilities should be compared after observing two deaths?

Let C = two cot deaths, M = two murders, and E = C or M = two babies died. Compare P(C | E) and P(M | E), conditioning explicitly on the evidence.

Why does a small probability of two cot deaths not by itself establish guilt?

Unlikely things happen all the time. Compare two cot deaths (C) with two murders (M). The slides’ illustrative calculation gives P(C)/P(M) ≈ 115, using similar, probably incorrect independence assumptions for both. A small P(C) alone does not establish guilt.