1/48
STAT0002 Week 3 — Chapter 3 lecture notes and Tuesday/Thursday Week 3 slides. Probability, types of events, conditional probability, addition and multiplication rules, and independence.
What is a random experiment?
An experiment whose outcome cannot be predicted with certainty.
What is the sample space S?
The set of all possible outcomes of a random experiment.
What is an event E?
A collection of outcomes, possibly just one outcome.
What is the relative frequency (frequentist) definition of probability?
The relative frequency with which the event occurs in the long run; the proportion of times that the event would occur in an infinite number of identical repeated experiments.
How is P(E) expressed using relative frequency?
P(E) is the limiting value of n(E)/n as n → ∞, supposing that this limit exists. n(E) denotes the number of times event E occurs in n experiments.
When choosing an individual at random from a population, what is the probability of a particular property?
The proportion of individuals in the population which have this property. All individuals are equally likely to be chosen.
Why can an observed relative frequency differ from the long-run probability?
It is based on finitely many random repetitions, so it can fluctuate. Repeating the same experiment many times typically gives a more stable relative frequency.

What are the basic properties of probability?
0 ≤ P(E) ≤ 1. If E is impossible then P(E) = 0; if E is certain then P(E) = 1. P(S) = 1 because the outcome must, by definition, be in the sample space.
In what sense are all probabilities conditional probabilities?
A probability is conditioned on the sample space S: defining S rules out anything not in S. For example, instead of P(C) we could write P(C | S). We should always bear in mind the sample space.
What do P(A,B), P(A and B), P(A ∩ B) and P(AB) mean?
The probability that both events A and B occur. P(A,B) = P(B,A).
What does P(A or B), or P(A ∪ B), mean?
The probability that A or B, or both, occur.
When are two events mutually exclusive (or disjoint)?
If they cannot occur together. Then P(A and B) = 0.
What is the addition rule for any two events A and B?
P(A or B) = P(A) + P(B) − P(A and B).

Why does the addition rule subtract P(A and B)?
Adding P(A) and P(B) counts the outcomes in both A and B twice. Subtracting P(A and B) counts them once.

How does the addition rule simplify for mutually exclusive events?
P(A or B) = P(A) + P(B), because P(A and B) = 0.
When are events A and B exhaustive?
If P(A or B) = 1.
What is the complement of an event A, and how is it denoted?
The event that A does not occur. It can be denoted not A, Ā or Aᶜ.
How are A and its complement related?
They are mutually exclusive and exhaustive. P(not A) = 1 − P(A).
What does the | sign mean in P(A | B)?
Conditional on, or given. P(A | B) is the probability of event A given event B.
What is the definition of conditional probability?
If B is an event with P(B) > 0, then for each event A, P(A | B) = P(A,B)/P(B).
How does conditioning on B change the sample space?
After conditioning on B, the relevant outcomes are the ones in B. Dividing by P(B)>0 makes the total probability over these outcomes equal to 1.
If P(A) > 0 and P(B) = 0, which version of the multiplication rule can be used?
P(A | B) is undefined because P(B) = 0. Instead use P(A,B) = P(B | A)P(A); here P(B | A) = 0 and P(A,B) = 0.
Do properties of probabilities also hold for conditional probabilities?
Yes. P(A | B) is a probability in a reduced sample space; for example, 0 ≤ P(A | B) ≤ 1.
Do P(A | B) and P(B | A) have the same meaning?
No. P(A | B) is the probability of A given B; P(B | A) is the probability of B given A.
What is an unconditional (or marginal) probability?
The overall probability of an event across the original sample space. In a contingency table it uses a row or column total divided by the grand total.
How do you test whether two events are independent from a table of counts?
Convert the relevant cell and marginal totals to probabilities using the same grand total. Compare P(A,B) with P(A)P(B) for the specified events.
How do the denominators differ for a joint and conditional probability from a table?
For P(A,B), divide the joint cell count by the grand total. For P(A ∣ B), divide the same cell count by the total for the conditioning event B, which must be positive.

When can you assign probability by counting outcomes?
If all elementary outcomes are equally likely, P(A) is the number of outcomes in A divided by the total number of possible outcomes. Otherwise add the probabilities of the outcomes in A.
What is the multiplication rule for two events?
If P(B) > 0, P(A,B) = P(A | B)P(B). Also, if P(A) > 0, P(A,B) = P(B | A)P(A).
Does the general multiplication rule require independence?
No. If P(B) > 0, P(A,B) = P(A | B)P(B) is always true. P(A,B) = P(A)P(B) only if A and B are independent.
What is the multiplication rule for three events?
P(A₁,A₂,A₃) = P(A₁)P(A₂ | A₁)P(A₃ | A₁,A₂), if all the conditional probabilities exist. The events can be ordered in other ways.
How does the multiplication rule generalise to n events?
P(A₁,…,Aₙ) = P(A₁)P(A₂ | A₁)P(A₃ | A₁,A₂)⋯P(Aₙ | Aₙ₋₁,…,A₁), provided all the conditional probabilities are defined.
What does P(A | B,C) mean?
The probability that event A occurs given that both events B and C have occurred. We condition on all events placed to the right of the | sign.
How can P(A | B,C) be expressed using joint probabilities?
P(A | B,C) = P(A,B,C)/P(B,C), provided P(B,C) > 0.
When are two events A and B independent?
If P(A,B) = P(A)P(B). Otherwise A and B are dependent events.
What does independence imply about conditional probability?
If P(B) > 0, P(A | B) = P(A): the occurrence of B does not affect the probability of A. Similarly, if P(A) > 0, P(B | A) = P(B).
Is independence symmetric?
Yes. If A is independent of B, then B is independent of A. A ⫫ B means “A and B are independent”.
Can the definition of independence apply to an event with zero probability?
Yes. If P(A) > 0 and P(B) = 0, then A and B are independent because P(A,B) = 0 = P(A)P(B).
Can mutually exclusive events with positive probabilities be independent?
No. For mutually exclusive events P(A,B) = 0, but if P(A) > 0 and P(B) > 0 then P(A)P(B) > 0.
Why can a conditional association in pooled data be misleading?
The mix of underlying groups may differ between the categories being compared. An overall conditional comparison can change when made within comparable groups.
When are three events A, B and C pairwise independent?
When all three hold: P(A,B) = P(A)P(B), P(A,C) = P(A)P(C), and P(B,C) = P(B)P(C).
When are three events A, B and C mutually independent?
They are pairwise independent and P(A,B,C) = P(A)P(B)P(C).
How do you express conditional probability as a ratio after fixing another event C?
If P(B,C)>0, then P(A ∣ B,C)=P(A,B ∣ C)/P(B ∣ C). This is the ordinary conditional probability formula within C.
When are A₁,…,Aₙ mutually independent?
For every r ∈ {2,…,n}, and every subset {C₁,…,Cᵣ} of {A₁,…,Aₙ}, P(C₁,…,Cᵣ) = P(C₁)⋯P(Cᵣ).
Is P(A,B,C) = P(A)P(B)P(C) alone sufficient for mutual independence?
The three-way product identity checks only the triple intersection. For mutual independence each pair also has to satisfy its own product identity.

How does the addition rule simplify for independent events?
For independent A and B, substitute P(A,B)=P(A)P(B) in the addition rule. This gives P(A∪B)=P(A)+P(B)−P(A)P(B).
Why is independence in repeated trials a modelling assumption?
Repeating the same procedure does not by itself make outcomes independent. A shared factor can change the probability of one outcome when another is known. Check the independence assumption before multiplying marginal probabilities.
How does conditional probability simplify if A implies B?
When A⊆B, A∩B=A. If P(B)>0, P(A ∣ B)=P(A)/P(B). If P(A)>0, P(B ∣ A)=1.
How should two competing explanations be compared after observing evidence E?
Compare their probabilities conditioned on E, e.g. P(H₁ ∣ E) and P(H₂ ∣ E). The small unconditional probability of one explanation alone does not tell you which explanation is more likely given E.