introduction to probability

variability of data

• measurement variability

• variability due to small changes in experimental conditions

uncertainty

corresponds to a lack of complete or perfect knowledge

causes of uncertainty

• incomplete observation of a system

• unpredictable variation

• simple lack of knowledge about the state of nature

  • current state that is imperfectly observed

  • future state that is the result of a study yet to be carried out

probability

numerical assessment of the chance of a particular event occurring, given a particular set of circumstances

experiment

some situation that generates multiple possible outcomes

aims of an experiment

• identify possible outcomes

• assign numerical value to collections of possible value to represent the chance that they will coincide with the actual outcome

• lay out the rules for how the numerical values can be manipulated

set (S)

• collection of individual elements (s)

• s ∈ S means s is one of the elements of S

finite set

• contains a finite number of elements

• S = {1,2,3,4,5}, S = {heads, tails}

countable set

• contains an infinite number of elements that can be listed

• S = {1,2,3,4,5,…}

uncountable set

• contains an infinite number of elements that cannot be listed

• S = {all values on a continuum between 0 and 100}

subset (A)

• contains some or all of the elements of S

  • some: A ⊂ S

  • some or all: A ⊆ S

• every element of A is also an element of S

  • s ∈ A → s ∈ S

empty set (∅)

subset that contains no elements

intersection (∩)

• collection of elements that are elements of both sets

  • s∈A∩B means s∈A and s∈B

• for any A,B⊆S:

  • A∩∅=∅

  • A∩S=A

  • A∩B⊆A

  • A∩B=B

union (∪)

• set of distinct elements that are either in A, or in B, or in both A and B

  • s∈A∪B means s∈A, or s∈B, or s∈A∩B

• for any A,B⊆S:

  • A∪∅=A

  • A∪S=S

  • A⊆A∪B

  • B⊆A∪B

complement (A^c)

• collection of elements of S that are not elements of A

  • s∈A^c means s∈S but s∉A

  • A∩A^c=∅

  • A∪A^c=S

• for any A, (A^c)^c=A

extensions of sets

• (A∩B)∩C=A∩B∩C

• (A∪B)∪C=A∪B∪C

• A∪(B∩C)=(A∪B)∩(A∪C)

• A∩(B∪C)=(A∩B)∪(A∩C)

partitions

• A₁, A₂,…,A_k

• pairwise disjoint (mutually exclusive)

  • A_j∩A_k=∅ for all j≠k

• exhaustive (the As cover the whole of S, but do not overlap)

• for every s∈S, s is an element of precisely one of the A_k

partitions for two subsets (A,B)

• A∩B

• A∩B^c

• A^c∩B

• A^c∩B^c

de Morgan’s Laws

• A^c∩B^c=(A∪B)^c

• A^c∪B^c=(A∩B)^c

setting of an experiment

any setting in which an uncertain consequence is to arise

sample space (S)

set of all possible outcomes of an experiment

sample points / outcomes

individual elements of the sample space S

event (A)

• collection of sample outcomes

• subset of sample space S

• individual sample outcomes are termed simple or elementary (E₁, E₂,…,E_k)

event terminology (for two events, A and B)

• A∩B occurs if and only if A occurs and B occurs

  • s∈A∩B

• A∪B occurs if A occurs or if B occurs, or if both A and B occur

  • s∈A∪B

• if A occurs, then A^c does not occur

• event S is termed the certain event

• event ∅ is termed the impossible event

rules of probability

• content cannot be negative

  • P(A) ≥ 0

• total content of S is standardized to be 1

  • P(S) = 1

• if A and B do not overlap, then their total content is the sum of their individual contents

  • P(A∪B) = P(A) + P(B)

  • extends to any number of disjoint events

corollaries of probability rules

• for any A, P(A^c) = 1 - P(A)

• P(∅) = 0

• for any A, P(A) ≤ 1

• for any two events A and B, if A⊆ B, then P(A) ≤ P(B)

• for two arbitrary events A and B, P(A∪B) = P(A) + P(B) - P(A∩B)

  • P(A₁∪A₂∪A₃) = P(A₁) + P(A₂) + P(A₃) - P(A₁∩A₂) - P(A₁∩A₃) - P(A₂∩A₃) + P(A₁∩A₂∩A₃)

Boole’s inequality

P(∪A_i) ≤ ∑P(A_i)

equally likely sample outcomes

• suppose that sample space S is finite, with N sample outcomes in total that are considered equally likely to occur

• for elementary events E₁, E₂,…,E_N, P(E_i)=(1/N) for i=1,…,N

• for any A⊆S, P(A) = (number of sample outcomes in A/number of sample outcomes in S) = n/N

frequentist definition of probability

• consider a finite sequence of N repeat experiments

• let n be the number of times out of N that A occurs

• define P(A) = lim_N→∞(n/N)

  • when N is really large, P(A)=n/N

rules for counting outcomes

• sequence of k operations, in which operation i can result in n_i possible outcomes, can result in n₁ x n₂ x … x n_k

• when selecting repeatedly from a finite set {1,2,…,N} we may select:

  • with replacement → each successive selection can be one of the original set, irrespective of previous selections

  • without replacement → the set is depleted by each successive election (one fewer outcome for each selection)

permutations (P_rⁿ)

• ordered arrangement of r distinct objects

• number of ways of ordering n distinct objects taken r at a time

• P_rⁿ = n x (n - 1) x (n - 2) x … (n - r + 2) x (n - r + 1) = (n!)/[(n - r)!]

  • where n! = n x (n -1) x … x 3 × 2 × 1

combinations

• number of subsets of size r that can be formed

  • choose r from n

• C_rⁿ = (n!)/[r!(n-r)!]

• P_rⁿ leaves (n - r) objects unselected

• if the order of selected objects is not important in identifying a combination, we must have that P_rⁿ = r! x C_rⁿ as there are r! equivalent combinations that yield the same permutation

binomial theorem

(a + b)ⁿ = ∑ [(n!)/[j!(n - j)!] a^jb^n-j]

binary sequences

• arise in many probability settings

• if we take “1” to indicate inclusion and “0” to indicate exclusion then we can identify combinations

• number of binary sequences of length n containing r 1s is (n!)/[r!(n-r)!]

combinatorial probability

• with the knowledge that event B occurs, we restrict the parts of the sample space that should be considered

  • we are certain that the sample outcome must lie in B

  • we can forget about those outcomes that are not in B

• for two events A and B, the conditional probability of A given B is:

  • P(A|B) = P(A∩B)/P(B)

  • we consider this only in cases where P(B) > 0

• if B≡S, then P(A|S) = P(A)

• if B≡A, then P(A|A) = 1

• P(A|B) ≤ P(B)/P(B) = 1