Introduction to Probability, Populations, and Sampling Methods

Fundamentals of Sets and Probability

• Set Definitions:

  • Set: A collection of objects, known as elements.

  • Subset: A set composed of elements belonging to another set.

  • Size: The total number of elements in a set.

• Experiment Definitions:

  • Experiment: A procedure repeated indefinitely, where a single occurrence is a trial.

  • Outcome: A possible result of an experiment.

  • Deterministic Experiment: An experiment with only a single possible outcome.

  • Random Experiment: An experiment with multiple outcomes defined by a long-run probability distribution.

  • Sample Space ($S$): The set of all possible outcomes. It can be discrete (countable outcomes) or continuous (measurable interval of outcomes).

  • Event: A subset of the sample space.

  • Disjoint Events: Events that share no common outcomes and cannot occur on the same trial.

Axioms and Rules of Probability

• Frequentist Probability: The proportion of times an event occurs across an infinite series of trials. For equally likely outcomes, $P(A) = \frac{\text{size of } A}{\text{size of } S}$.

• Axioms of Probability:

  • Axiom I: $0 \le P(A) \le 1$ for any event $A$.

  • Axiom II: $P(S) = 1$.

  • Axiom III: For any two disjoint events $A$ and $B$, $P(A \text{ or } B) = P(A) + P(B)$.

• Core Rules:

  • Complement Rule: $P(\sim A) = 1 - P(A)$.

  • General Addition Rule: $P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$.

  • Benford's Law: Calculates the probability of first digits $d$ in naturally occurring datasets as $P(d) = \log_{10}\left(1 + \frac{1}{d}\right)$.

Conditional Probability and Independence

• Conditional Probability: The probability of event $B$ occurring given that event $A$ has occurred (where $P(A) > 0$):   $P(B|A) = \frac{P(A \text{ and } B)}{P(A)}$

• General Multiplication Rule:   $P(A \text{ and } B) = P(A) \times P(B|A) = P(B) \times P(A|B)$

• Independence: Two events $A$ and $B$ are independent if $P(B|A) = P(B)$ or $P(A|B) = P(A)$.

  • For independent events: $P(A \text{ and } B) = P(A) \times P(B)$.

  • For multiple independent events: $P(A \text{ and } B \text{ and } C \dots) = P(A) \times P(B) \times P(C) \times \dots$

Variables and Population Concepts

• Core Definitions:

  • Individual: An object of interest.

  • Population: A set of individuals with population size $N$.

  • Sample: A subset of the population observed with sample size $n$.

  • Variable: An attribute of an individual.

  • Value: The category or number assigned to an individual by a variable.

• Types of Variables:

  • Qualitative (Categorical):

  • Nominal: Categories with no natural ordering (e.g., state of residence).

  • Ordinal: Categories with a natural ordering (e.g., coffee size).

  • Quantitative (Numerical):

  • Discrete: Countable numeric values (e.g., number of defects).

  • Continuous: Measurable numeric values (e.g., weight in kilograms).

Sampling Methods and Bias

• Sampling Designs:

  • Simple Random Sample (SRS): A sample of size $n$ selected such that every set of size $n$ has an equal probability of selection.

  • Multistage Sampling: Sampling that involves dividing a population into groups, then randomly selecting groups across multiple successive stages.

  • Stratified Random Sampling: Dividing a population into defined groups (strata), then taking a proportional SRS from each group.

• Sources of Sampling Bias:

  • Convenience Sample: Selecting readily accessible individuals, resulting in undercoverage.

  • Voluntary Response Sample: Relying on individuals who choose to respond, leading to nonresponse bias.

  • Response Bias: Systemic factors causing inaccurate responses, including social desirability bias and wording effects.

Fundamentals of Sets and Probability

• Set Definitions:

  • Set: A collection of distinct objects, known as elements (e.g., $S = \{1, 2, 3, 4, 5, 6\}$).

  • Subset: A set composed entirely of elements belonging to another set (e.g., $\text{California}$ within the set of U.S. West Coast states).

  • Size: The total number of elements contained within a set.

• Experiment and Outcome Definitions:

  • Experiment: A procedure or set of procedures that can be repeated indefinitely.

  • Trial: A single occurrence or execution of an experiment.

  • Outcome: A single possible result of an experiment.

  • Deterministic Experiment: An experiment that yields only a single possible outcome every time it is performed.

  • Random Experiment: An experiment with multiple possible outcomes defined by a long-run probability distribution.

  • Sample Space ($S$): The set of all possible outcomes of an experiment.

  • Discrete Sample Space: A sample space consisting of countable outcomes (e.g., rolling a six-sided die where $S = \{1, 2, 3, 4, 5, 6\}$, or flipping a coin three times where $S = \{HHH, HHT, HTH, THH, HTT, THT, TTH, TTT\}$).

  • Continuous Sample Space: A sample space composed of a continuous, measurable interval of outcomes (e.g., generating a real number between $0$ and $1$ where $S = [0, 1]$, or measuring temperature).

  • Event: Any subset of outcomes from the sample space (e.g., rolling an even number $B = \{2, 4, 6\}$, or flipping no heads and no tails $Z = \emptyset$).

  • Disjoint (Mutually Exclusive) Events: Events that share no common outcomes and therefore cannot occur on the same trial of an experiment (e.g., rolling an even number $B = \{2, 4, 6\}$ and rolling an odd number $C = \{1, 3, 5\}$).

Axioms and Rules of Probability

• Frequentist Probability: Defines the probability of an event as the proportion of times it occurs across an infinite series of repeated trials.

  • For sample spaces with equally likely outcomes:     $P(A) = \frac{\text{size of } A}{\text{size of } S}$

  • Bayesian Probability: An alternative theoretical approach interpreting probability as a measure of belief or state of knowledge.

• Probability Models and Variables:

  • Probability Distribution: A description detailing the values a mathematical variable can take and how frequently it takes them.

  • Random Variable: A numerical outcome of a random experiment, often represented by uppercase letters such as $X$ or $Y$.

  • Probability Model: A mathematical representation consisting of a sample space $S$ together with a defined method for assigning probabilities to events.

• The Three Axioms of Probability:

  • Axiom I: For any event $A$, the probability satisfies $0 \le P(A) \le 1$.

  • Axiom II: The total probability of the entire sample space $S$ is equal to $1$, written as $P(S) = 1$.

  • Axiom III: For any two disjoint events $A$ and $B$:     $P(A \text{ or } B) = P(A) + P(B)$

• Core Rules and Theorems:

  • Mutually Disjoint Extension: If events $A$, $B$, $C$, etc. are mutually disjoint, then:     $P(A \text{ or } B \text{ or } C \dots) = P(A) + P(B) + P(C) + \dots$

  • Complement Rule: The probability that event $A$ does not occur is given by:     $P(\sim A) = 1 - P(A)$

  • General Addition Rule: For any two events $A$ and $B$ (whether disjoint or overlapping):     $P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$

  • Benford's Law: Calculates the probability distribution of the first non-zero digit $d$ in naturally occurring datasets:     $P(d) = \log_{10}\left(1 + \frac{1}{d}\right)$

Conditional Probability and Independence

• Conditional Probability: The probability that event $B$ occurs given that event $A$ has already occurred (valid when $P(A) > 0$):   $P(B|A) = \frac{P(A \text{ and } B)}{P(A)}$

• General Multiplication Rule: The probability that both events $A$ and $B$ occur simultaneously:   $P(A \text{ and } B) = P(A) \times P(B|A) = P(B) \times P(A|B)$

• Independence:

  • Two events $A$ and $B$ are independent if the occurrence of one does not affect the probability of the other:     $P(B|A) = P(B) \quad \text{or} \quad P(A|B) = P(A)$

  • Multiplication Rule for Independent Events:     $P(A \text{ and } B) = P(A) \times P(B)$

  • Extension to Multiple Independent Events:     $P(A \text{ and } B \text{ and } C \dots) = P(A) \times P(B) \times P(C) \times \dots$

• Event Relationship Summary:

  • For disjoint events, "or" maps to addition: $+$

  • For independent events, "and" maps to multiplication: $\times$

Variables and Population Concepts

• Core Definitions:

  • Individual: An object or subject of interest in a study.

  • Population: The complete collection of all individuals being studied, with total population size denoted as $N$.

  • Sample: A subcollection or subset of individuals selected from the population to be observed, with sample size denoted as $n$.

  • Variable: A specific characteristic or attribute of an individual.

  • Value: The numerical quantity or categorical label assigned to an individual for a specific variable.

  • Datum (Observation): The recorded value of a variable for a single individual (plural: data).

  • Dataset: A collected set of data observations.

• Branches of Statistics:

  • Descriptive Statistics: Focuses on gathering, representing, summarizing, and comparing sample data to describe its key properties.

  • Inferential Statistics: Uses sample data to make estimates, calculate confidence intervals, and perform hypothesis testing regarding the overall population.

• Types of Variables:

  • Qualitative (Categorical): Assigns individuals to descriptive categories.

  • Nominal: Categories with no natural, logical ordering (e.g., state of residence, zip code, passport status).

  • Ordinal: Categories with a clear, natural ordering (e.g., coffee sizes like Small/Medium/Large, education levels).

  • Quantitative (Numerical): Assigns numeric values to individuals.

  • Discrete: Countable numeric values (e.g., number of dependents, number of product defects).

  • Continuous: Measurable numeric values along a continuous scale (e.g., beak length of a bird, weight in kilograms).

• Distributions:

  • Population Distribution: The values taken by a variable across the entire population and the frequency of each value.

  • Sample Distribution: The values taken by a variable within a specific sample and the frequency of each value.

Sampling Methods and Bias

• Sampling Designs:

  • Simple Random Sample (SRS): A sample of size $n$ chosen such that every possible group of size $n$ from the population has an equal chance of selection.

  • Random Digit Table Method: Assign equal-length numerical labels to all individuals, select a starting row, and read off non-overlapping digit groups of length matching the label size. Skip unused numbers and duplicates until $n$ individuals are chosen.

  • Multistage Sampling: Divides the population into groups, randomly selects a subset of those groups, and then performs further stages of grouping and random selection (e.g., selecting states, then municipalities, then citizens).

  • Stratified Random Sampling: Divides the population into distinct homogeneous subgroups (strata) based on specific characteristics, then selects a proportional SRS from each stratum (e.g., sampling undergraduate and graduate students in proportion to their class representation).

• Sources of Sampling Bias:

  • Bias: Systemic favoring of certain outcomes or values over others during the sampling process.

  • Convenience Sample: Selecting individuals who are easiest to reach, leading to undercoverage (excluding parts of the population, such as surveying only college classmates to estimate high school SAT scores).

  • Voluntary Response Sample: Allowing individuals to choose whether to participate, leading to nonresponse bias (e.g., using bait that only a specific fish species eats).

  • Response Bias: Factors causing respondents to give inaccurate or misleading answers during surveys or polls.

  • Social Desirability Bias: Tendency of respondents to give socially acceptable answers rather than truthful ones.

  • Wording Effects: Phrasing survey questions in a biased manner to sway responses toward a desired result.

### Fundamentals of Sets and Probability - **Core Set Theory Definitions**: - **Object**: Anything that is defined mathematically or conceptually. - **Set**: A well-defined collection of distinct objects, known as elements (e.g., $S = \{1, 2, 3, 4, 5, 6\}$). - **Subset**: A set composed entirely of elements belonging to another set (e.g., $\{\text{California}\}$ within the set of U.S. West Coast states). - **Size (Cardinality)**: The total count of elements contained within a set. - **Empty Set ($\emptyset$)**: A set containing no elements, representing an impossible outcome event. - **Experiment and Outcome Concepts**: - **Experiment**: A structured procedure or process that can be repeated indefinitely. - **Trial**: A single execution or occurrence of an experiment. - **Outcome**: A specific possible result generated by a single trial of an experiment. - **Deterministic Experiment**: An experiment that yields the exact same single outcome every time it is performed. - **Random Experiment**: An experiment with multiple possible outcomes whose results vary unpredictably across trials but exhibit a long-run probability distribution. - **Sample Space ($S$)**: The exhaustive set of all possible outcomes of an experiment. - **Discrete Sample Space**: A sample space composed of distinct, countable outcomes (e.g., rolling a die where $S = \{1, 2, 3, 4, 5, 6\}$). - **Continuous Sample Space**: A sample space consisting of an uncountably infinite, measurable interval of real numbers (e.g., measuring temperature or picking a real number in $S = [0, 1]$). - **Event**: Any subset of outcomes from the sample space $S$. - **Disjoint (Mutually Exclusive) Events**: Events that share no outcomes in common ($A \cap B = \emptyset$) and therefore cannot occur on the same trial. ### Axioms and Rules of Probability - **Theoretical Approaches to Probability**: - **Frequentist Probability**: Defines the probability of an event as the limiting relative frequency of its occurrence over an infinite series of independent trials. For equally likely outcomes: $P(A) = \frac{\text{size of } A}{\text{size of } S}$. - **Bayesian Probability**: Interprets probability as a measure of belief, certainty, or state of knowledge revised in light of new evidence. - **Probability Distributions and Models**: - **Random Variable ($X, Y$)**: A quantitative variable whose numerical value is determined by the outcome of a random experiment. - **Probability Distribution**: A function or table describing all values a random variable can take and the corresponding probabilities of taking those values. - **Probability Model**: A mathematical framework consisting of a sample space $S$ paired with a valid assignment rule for probabilities of events. - **The Three Axioms of Probability**: - **Axiom I (Non-negativity)**: For any event $A$, $0 \le P(A) \le 1$ - **Axiom II (Normalization)**: The total probability of the entire sample space is $1$, written as $P(S) = 1$ - **Axiom III (Additivity for Disjoint Events)**: For any two disjoint events $A$ and $B$: $P(A \text{ or } B) = P(A) + P(B)$ - **Fundamental Probability Theorems**: - **Mutually Disjoint Extension**: For any collection of mutually disjoint events $A, B, C, \dots$: $P(A \text{ or } B \text{ or } C \dots) = P(A) + P(B) + P(C) + \dots$ - **Complement Rule**: The probability that event $A$ does not occur is $P(\sim A) = 1 - P(A)$ - **General Addition Rule**: For any two events $A$ and $B$ (disjoint or overlapping): $P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$ - **Benford's Law**: The empirical probability distribution of the first non-zero digit $d \in \{1, 2, \dots, 9\}$ in naturally occurring datasets: $P(d) = \log_{10}\left(1 + \frac{1}{d}\right)$ ### Conditional Probability and Independence - **Conditional Probability**: The revised probability of event $B$ given that event $A$ has already occurred (valid when $P(A) > 0$): $P(B|A) = \frac{P(A \text{ and } B)}{P(A)}$ - **General Multiplication Rule**: The probability that both events $A$ and $B$ occur together: $P(A \text{ and } B) = P(A) \times P(B|A) = P(B) \times P(A|B)$ - **Independence**: - Two events $A$ and $B$ are statistically independent if knowledge of one provides no information about the probability of the other: $P(B|A) = P(B) \quad \text{or} \quad P(A|B) = P(A)$ - **Multiplication Rule for Independent Events**: $P(A \text{ and } B) = P(A) \times P(B)$ - **Extension to Multiple Independent Events**: $P(A \text{ and } B \text{ and } C \dots) = P(A) \times P(B) \times P(C) \times \dots$ ### Variables and Population Concepts - **Core Statistical Definitions**: - **Individual**: A single object or subject of interest in a study. - **Population**: The complete collection of all individuals under study, with size denoted by $N$ - **Sample**: A representative subset of individuals selected from the population, with size denoted by $n$ - **Variable**: A specific characteristic or attribute recorded for each individual. - **Value**: The specific numeric quantity or category assigned to an individual for a variable. - **Datum (Observation)**: The recorded value of a variable for a single individual (plural: data). - **Dataset**: A structured collection of recorded observations. - **Branches of Statistics**: - **Descriptive Statistics**: Methods for organizing, summarizing, displaying, and describing sample data properties. - **Inferential Statistics**: Methods for drawing conclusions, making estimations, and testing hypotheses about a population based on sample data. - **Classification of Variables**: - **Qualitative (Categorical)**: Variables assigning individuals to descriptive categories. - **Nominal**: Categorical variables with no natural or logical order (e.g., state, zip code, passport status). - **Ordinal**: Categorical variables with a natural order or ranking (e.g., coffee size, education level). - **Quantitative (Numerical)**: Variables assigning numeric values representing quantities. - **Discrete**: Quantitative variables with countable values or jumps (e.g., number of defects, number of dependents). - **Continuous**: Quantitative variables with infinitely fine, measurable continuous values (e.g., weight, height, beak length). ### Sampling Methods and Sources of Bias - **Sampling Techniques**: - **Simple Random Sample (SRS)**: A sampling design where every subset of size $n$ from the population has an equal chance of being chosen. - **Random Digit Table Method**: Labelling every population member with equal-digit numbers, choosing a starting row, reading non-overlapping digit groups, and selecting non-duplicate labels. - **Multistage Sampling**: Sampling carried out in successive hierarchical stages by randomly selecting groups within groups. - **Stratified Random Sampling**: Partitioning the population into non-overlapping homogeneous strata based on key characteristics, then selecting a proportional SRS from each stratum. - **Sampling Bias and Fallacies**: - **Bias**: Any systematic deviation in a sampling procedure that favors certain outcomes over others. - **Convenience Sampling**: Selecting individuals who are easiest to reach, leading to **undercoverage** (omitting segments of the population). - **Voluntary Response Sampling**: Allowing individuals to self-select into the sample, leading to **nonresponse bias**. - **Response Bias**: Systemic factors causing inaccurate answers during data collection. - **Social Desirability Bias**: Tendency of respondents to give socially acceptable answers rather than truthful ones. - **Wording Effects**: Question phrasing that steers respondents toward a particular answer. - **Survivorship Bias**: Concentrating only on subjects that passed a selection process or survived an event while unintentionally ignoring those that did not (e.g., analyzing bullet damage only on surviving combat aircraft).