Comprehensive Study Notes – Introduction to Statistics & Probability Theory

Introduction, Philosophy & Scope

• Education aim: teach students how to think, not what to think – statistical reasoning epitomises this goal.
• Statistics = ensemble of methods to:
  • Plan experiments
  • Obtain data
  • Organise, summarise & present data
  • Analyse & interpret results
  • Draw defensible conclusions
• Viewed as the “art / science of decision-making in the face of uncertainty.”

Motivating Metaphors & Illustrative Stories

• Data Compression Analogy
  • Long word “invertebrate” $\to$ substitute with “zqz” → small gain because word is rare.
  • Common word “then” $\to$ “zqz” → large overall file-size reduction.
    ⇒ Insight: good statistical modelling finds frequent patterns to achieve maximal efficiency.
• King Solomon & Two Mothers
  • Decision about true mother depends on resemblance (eyes vs noses) → cannot be certain, but probability of correct guess can be evaluated.
  • Sparks debate between “eyeists” vs “nosists” – mirrors model selection in statistics.

Populations, Samples & Sampling Motivation

• Population: complete group of units we wish to study (e.g. all 15-year-old girls).
• Sample: subset actually examined; must be “representative.”
• Random Sample: every element of population has equal chance of selection ⇒ reduces systematic bias.
• Why sample instead of census?
  1. \text{Time & cost savings}
  2. Avoid destructive testing (e.g. blood test destroys blood vial).
  3. Population may be inaccessible or infinite.

Random Experiments & Outcomes

• Random Experiment: repeatable process yielding an observation; outcome cannot be predicted with certainty.
  • Examples: tossing a coin, rolling a die, drawing a ball.
• Sample Space $S$: set of all possible outcomes.
  • Toss die: $S={1,2,3,4,5,6}$ (discrete, finite).
  • Classify outcome as even/odd: alternative sample space less informative.
  • Types:
    • Finite
    • Countably infinite (e.g. $S={1,2,3,\ldots}$)
    • Non-countably infinite (continuum $0\le x\le1$).
    • Discrete = finite or countably infinite; Nondiscrete = continuous.

Events & Set-Theoretic Language

• Event $A\subseteq S$: collection of outcomes.
• Elementary event: single outcome.
• Certain event $S$ & Impossible event $\varnothing$.
• Operations
  • Union: $A\cup B$ = “$A$ or $B$ or both.”
  • Intersection: $A\cap B$ = “both $A$ and $B$.”
  • Complement: $A' = S-A$.
  • Difference: $A-B = A\cap B'$.
• Mutually Exclusive: $A\cap B = \varnothing$ ⇒ cannot co-occur.
• Subset notation
  • $A\subset B$ (proper), $A=B$, $B\not\subset A$, $\varnothing\subset X$.
• Roster & Set-builder Examples (from slides 18-21)
  • $C={1,2,3,4,5,6},\ D={4,5,6,7,8,9}$ ⇒
    • $C\cup D={1,2,3,4,5,6,7,8,9}$
    • $C\cap D={4,5,6}$.
  • Word problems on unions/intersections of numeral descriptions.

Concept of Probability

• Quantifies uncertainty on $[0,1]$ scale (0%–100%).
• $P(A)=1$ ⇒ certainty; $P(A)=0$ ⇒ impossibility; $P(A)=\tfrac14$ ⇒ 25% chance.
• Odds: if $P=\tfrac14$, odds against are $3:1$.

Approaches to Estimation

1. Classical (Laplace)
   • Equally likely assumption.
   • $P(A)=\frac{h}{n}$ where $h$ favourable, $n$ total.
2. Frequency / Empirical
   • After $n\gg1$ trials, $P(A)\approx\frac{h}{n}$.
3. Axiomatic (Kolmogorov)
   • Resolves vagueness via formal postulates.

Kolmogorov Axioms

Given class $\mathcal C$ of measurable events and probability function $P$:

1. $0\le P(A)\le1$ for all $A$.
2. $P(S)=1$.
3. For sequence of mutually exclusive events $A<em>1,A</em>2,\ldots$,
   $P\Big(\bigcup<em>{i}A</em>i\Big)=\sum<em>i P(A</em>i)$ (countable additivity).
   • Finite version yields $P(A\cup B)=P(A)+P(B)-P(A\cap B)$.

Derived Properties

• Complement: $P(A')=1-P(A)$.
• Monotonicity: if $A\subseteq B$ then $P(A)\le P(B)$.
• Inclusion-exclusion (3 sets):
  $P(A\cup B\cup C)=P(A)+P(B)+P(C)-P(A\cap B)-P(A\cap C)-P(B\cap C)+P(A\cap B\cap C).$

Independence & Conditional Probability

Independent Events

• Definition: $P(B|A)=P(B)\iff P(A\cap B)=P(A)P(B)$.
• Intuition: knowledge of $A$ doesn’t alter belief about $B$.
• Multiple independent events:
  $P(A<em>1\cap A</em>2\cap\ldots\cap A<em>n)=\prod</em>{i=1}^n P(A_i).$
• Spinner example (slide 29):
  • Even numbers: $P=\tfrac36$; vowels: $P=\tfrac15$;
    $P(\text{even}\cap\text{vowel})=\tfrac36\times\tfrac15=\tfrac{1}{10}.$

Dependent Events

• Occurrence of one alters probability of the other.
• Conditional probability: $P(B|A)=\frac{P(A\cap B)}{P(A)}$.
• Product rule (two events):
  $P(A\cap B)=P(A)\,P(B|A).$
• Three dependent events (sandwich example):
  $P(A\cap B\cap C)=P(A)\,P(B|A)\,P(C|A\cap B)$
  $=\frac{10}{10}\times\frac{9}{10}\times\frac{8}{10}=\frac{18}{25}=0.72.$
• Pen example: 6 black, 8 blue. Without replacement:
  $P(\text{black}<em>1)=\tfrac6{14},\;P(\text{black}</em>2|\text{black}_1)=\tfrac5{13}$
  $\Rightarrow P(\text{both black})=\tfrac{30}{182}=\tfrac{15}{91}\approx0.165.$
  (Students prompted to compute.)

Worked Problem Themes (Slides 38–40)

• Two-dice analysis: building full $36$-element sample space; compute $P(A),P(B),P(A\cap B),P(A\cup B).$
  • Independence → check $P(A\cap B)=P(A)P(B)$.
  • Mutual exclusivity → check $A\cap B\ne\varnothing$.
• Venn-diagram applications for exam failures and course enrolments; inclusion-exclusion used to derive “at least one” probabilities.

Descriptive Statistics – Position Measures

• Quartiles $x<em>{0.25},x</em>{0.50},x_{0.75}$
  • Lower quartile: 25 % below.
  • Median: 50 %.
  • Upper quartile: 75 % below.
  • Inter-quartile range $\text{IQR}=x<em>{0.75}-x</em>{0.25}$; semi-IQR $=\tfrac{\text{IQR}}2.$
• Deciles $x<em>{0.1},x</em>{0.2},\ldots,x_{0.9}$ divide data into 10 regions.
• Percentiles $x<em>{0.01},\ldots,x</em>{0.99}$ – 80th percentile: 80 % data below.
• Note: 50th percentile = median.

Statistical Terminology & Data Types

• Census: data from full population.
• Parameter: numerical measure describing population.
• Statistic: numerical measure describing sample.
• Quantitative data: counts/measurements (e.g. height, income).
• Qualitative (categorical) data: categories distinguished by attribute (e.g. colour, brand).

Ethical & Practical Considerations

• Representativeness of samples critical to avoid misleading inferences.
• Destructive testing poses ethical & cost dilemmas; sampling mitigates harm.
• Decision-making in uncertain environments (e.g. Solomon’s judgement) involves balancing risk & evidence – statistical literacy empowers better societal choices.

Inspirational Anecdote

• Bill Gates quote: “I failed in some subjects… friend passed all… he is engineer at Microsoft – I own Microsoft.”
  – Highlights non-deterministic pathways; statistics provides probabilities, not certainties.

Connections & Foundations

• Builds on prior knowledge of set theory and logic → essential for probability.
• Forms basis for upcoming topics: random variables, distributions, estimation & hypothesis testing.

Key Formulae Summary (Cheat-Sheet)

• Classical probability: $P(A)=\frac{h}{n}$.
• Complement: $P(A')=1-P(A)$.
• Inclusion-Exclusion (two sets): $P(A\cup B)=P(A)+P(B)-P(A\cap B)$.
• Conditional: $P(B|A)=\frac{P(A\cap B)}{P(A)}$.
• Independent events: $P(A\cap B)=P(A)P(B).$
• Dependent events (3): $P(A\cap B\cap C)=P(A)P(B|A)P(C|A\cap B).$
• IQR: $\text{IQR}=x<em>{0.75}-x</em>{0.25}$.