PROBABILITY & STATISTICS

Probability

measure of one’s belief in the possible occurence of an event

Random/Stochastic Events

• cannot be predicted with certainty

• have stable relative frequencies over long period of trials

Relative Frequency

The number of p occurences observed in n trials, divided by n, when n > 0.

• as n increases, the limit of this may approach the value of the probability, making it merely an estimate

Population

set of values that can be generated as the scenario is performed ad infinitum

Hypothesis

an inferred statement made to test a point; may seek to contradict this using observation/experimentation

Highly Improbable

a result that is very unlikely though not impossible; eg. an astronomically low p-value

Set

a collection of distinct objects/values that share a common property

Union

the elements in either or both of two sets

Intersection

the set of values two sets share in common

Compliment

the set of points outside of a specific set

Pairwise Disjoint

2 sets that are mutually exclusive; share nothing in common.

Distributive Laws

Intersections distribute over unions.

• A (intersects) (B U C) <=> (intersection of A & B) U (intersection of A & C)

De Morgan’s Laws

The compliment of intersections is the union of compliments (or vice versa: the compliment of unions is the intersection of compliments)

• (A U B)^c = intersection of A^c & B^c

Experiment

A process for which an observation is made

Event

An outcome of an experiment

Simple event

An event that cannot be broken down any further

• has unique sample points

• are disjoint to each other if an experiment is only performed once

Compound event

An event that CAN be broken down further into more specific events; “broad event“

Sample point

BIJECTIVE (one-to-one and onto) with a sample event

Sample space

The set of all possible points

Countable

True when there exists a set that has a bijection with itself + natural numbers

Discrete sample space

Exists if the cardinality is either finite or countable

Discrete event

any subset of discrete sample space

• in other words: collection of sample points

Discrete probabilistic model

Assigns numerical probabilities to each simple event found in the discrete sample space S

Sample-point method

Finds probability of an event A in sample space S, when S is at most countable.

Steps of sample-point method

1) DEFINE

• sample space - by listing sample events

• simple events

• experiments

2) ASSIGN PROBABILITIES

• all probabilities must sum to 1 and be less than or equal to 0 separately

3) DEFINE A

• the union of all applicable simple events

• test each point

4) FIND P(A)

• sum up all the simple events in A to get the probability

“mn” rule

With m elements in 1 set and n elements in another, this rule makes it possible to form mn pairs, containing 1 element from each group

• can be extended past 2 sets

n!/(n-r)!

The number of ways to order n distinct objects taken r at a time

n!/(n₁!n₂!…n_k)

The number of ways to order n distinct objects into k groups in order - where every object is in exactly 1 group

n!/r!(n-r)!

The number of unordered subsets of size r that can be formed with n objects WITHOUT REPLACEMENT

C^n_r (n choose r)

The number of combinations of n objects taken r at a time WITHOUT REPLICATION

Unconditional Probability

Ignoring other factors, the fraction of p (successful event) over period observed

Conditional Probability

The probability of an event A given that an event B has also occured, P(A|B) (A given B).

• P(A|B) = P(A and B)/P(B)

Independence

The occurence of an event A is UNAFFECTED by the occurence of an event B

• P(A|B) = P(A) or P(B)

• P(A or B) = P(A) * P(B)

Multiplicitive Law

For any 2 events A and B, P(A or B) = P(A)P(A|B) = P(B)P(B|A)

• if independent, = P(A) * P(B)

• with 3 events, = P(A or B or C) = P(A or B)P(C | A or B)

Additive Law

For any 2 events A and B… P(A and B) = (P(A) + P(B)) - P(A or B)

• if disjoint, = P(A) + P(B) only

Complement Rule

For any event A, P(A) = 1 - P(A^c)

Event Composition Method

A way to find the probability of a compound event

Steps of Event Composition Method

1) Define experiment

2) Visualize & identify sample points

3) Write equation expressing event A as a composition of 2+ simple events

4) Apply laws of probability to find P(A)

Decomposition of Events

One of probable options on nth draw.

• eg. B₁ is best applicant drawn on the 1st draw. B₂ is the worst drawn on the 2nd. P(B) = B₁ U B₂

• B₁ and B₂ are disjoint events

Partition

For positive integer k and the collection of sets B₁, B₂,…B_k being disjoint, these sets are a subset of the sample space and partition the sample space

Law of Total Probability

(sum from i=1 to k) P(A|B_i)P(B_i)

• given partition of S with probabilities > 0

Baye’s Rule

P(A|B_j)P(B_j) / P(A|B_i)P(B_i)

• given partition of S for which P(B_i) > 0

Random Variable

A real value function in which the domain is a sample space

Random Sample

Given population N and sample size n, each [N choose n] possible sample has the same probability of getting selected.

Discrete Random Variable

A random variable with a range that is at most countable

Probability Distribution

The collection of the probabilities of each value in a random variable

Traits of Probability Distributions

1) shown as a table, formula, or graph, all of which must provide p(y) = P(Y = y) for all y

2) P(y) > 0 for all y and {y : p(y) > 0} is at most countable for discrete Y

3) any y with p(y) not explicitly assigned has a probabilty of 0

Probability Function

can be represented by the function p(y) that assigns probability values to each value y, given its probability of a random variable Y taking on the value y is P(Y = y) = p(y)

Expected Value

• If given discrete random variable Y and probability distribution p(y), is: (sum of all y) y * p(y) if series is absolutely convergent

• If given p(y) that is an approximate characterization of the population frequency distribution, is the population mean μ

• If given real-value function g(Y), is (sum of all y) g(y) * p(y) if series is absolutely convergent

• if given constant C, E(C) = C; if given C and real-value function, is C * E(g(Y))

Variance

Average (deviation from mean)², given by the formula (Y - μ)² / n (total values)

• This is E[(Y - μ)²] given E(Y) = μ

Standard Deviation (σ)

Given E(Y) = μ, is the positive square root of the variance V(Y)

Population Variance

Is given by σ² (standard deviation squared) when E(Y) = μ

p

probability of success

q

probability of failure

s

number of successes

f

number of failures

Binomial Experiment

• consists of a fixed number n identical trials

• has 2 outcomes (success/failure)

• outcomes of success and failure are independent

• probabilities of p and q remain constant

• random variable of interest is s, number of successes

Geometric Random Variable

the number in which the first success occurs

• this is typically the last trial; experiment ends after success

Geometric Probability Distribution

in which random variable Y is such where p(y) = q^y-1 * p and 0 < y < 1.

• variance = (1-p) / p²

• mean + E(Y) = 1/p