Random Sampling, Probability Theory, and Combinatorial Counting Rules
Foundations of Random Sampling
Definition of a Simple Random Sample:
A simple random sample of items from a population requires the satisfaction of two core conditions:
Condition (a): Every member of the population possesses the exact same probability of being included in the sample.
Condition (b): The members of the sample are selected completely independently of one another.
Methods and Tools for Generating Random Samples:
Obtaining an authentic random sample in practice can present significant operational challenges.
Standard tools and methods utilized to generate random samples include:
Calculators with built-in pseudorandom generator functions.
Computer algorithms designed for automated random selection.
Research Randomizer (an online utility designed for instant random sampling and random assignment).
Standardized random number tables.
Purpose of Random Sampling in Statistical Inference:
In order to draw valid inferences about a broader population from an observed sample, the sample must provide an accurate and unbiased representation of the population.
Implementing a formal random sampling model is the most effective approach to ensure representative sampling.
Once a representative random sample is established, researchers can validly apply the rules of probability to draw mathematical inferences about the target population.
Probability Definitions and Frameworks
Fundamental Bounds and Definition of Probability:
Probabilities are real numerical values defined on the closed interval between and , inclusive (), that quantify the likelihood of a specific event occurring.
Probabilities approaching signify that the event is highly likely to take place.
Probabilities approaching signify that the event is highly unlikely to take place.
A probability value of precisely denotes certainty: the event will occur.
A probability value of precisely denotes impossibility: the event will not occur.
Equally Likely Outcomes Model:
When an experiment has a finite number of possible outcomes that are all equally likely to occur, the probability of an event is formally defined as:
In this formulation:
represents the total number of favorable outcomes for which the event occurs.
represents the total number of all possible outcomes within the sample space.
Simple Probability:
Simple probability denotes the likelihood of occurrence of a single, individual simple event, denoted symbolically as .
Three Approaches to Determining Probability:
Determining probability varies depending on the specific application, environment, and constraints under study:
Subjective Probability: Probabilities based on personal judgment, qualitative assessment, beliefs, or subjective expectations rather than mathematical proofs or frequency data.
A Priori Classical Probability: Probabilities derived deductively from prior knowledge and theoretical understanding of the physical or mathematical process generating the outcomes.
Empirical Classical Probability: Probabilities established inductively from observed relative frequencies within empirical sample data or experimental trials.
Classical, Empirical, and Subjective Probability Applications
A Priori Classical Probability (Marginal Probability):
Computed purely from structural understanding of the system:
This measure is also referred to as marginal probability when focused on a single categorical dimension.
Standard Playing Card Deck Application:
Objective: Determine the probability of randomly drawing a face card from a standard deck of playing cards.
A standard deck contains face cards (four Jacks, four Queens, and four Kings) out of total cards.
Calculation:
Empirical Classical Probability via Contingency Tables:
Empirical probability is calculated from observed counts structured in a two-way contingency table.
Survey Data Contingency Table ( individuals):
| Category | Eats Junk Food | Does Not Eat Junk Food | Total | | :--- | :--- | :--- | :--- | | Does not Watch Sports on TV | | | | | Watches Sports on TV | | | | | Total | | | |
* *Empirical Calculations from Table*:
* Probability that a randomly selected individual does not watch sports on TV:
* Probability that a randomly selected individual eats junk food: (Note: Listed in raw source text as , which contains a typographical denominator for the grand total sample size of , yielding the value .)
Real-World Context: Gambling Paradigms:
Different forms of gambling correspond directly to the three probability processes:
Subjective Probability: Governs wagering on competitive sports such as horse racing and football games, where external factors, expert consensus, and fluctuating conditions dictate the odds.
Empirical Classical Probability: Governs strategic games such as poker to a certain extent, where players calculate frequencies based on observed play styles and empirical card distributions.
A Priori Classical Probability: Governs mechanical casino games such as rolling dice and roulette wheels, where the sample space and exact probabilities are dictated entirely by geometry and physics.
Joint Probability and Marginal Decompositions
Joint Probability:
Joint probability measures the chance that two distinct events take place concurrently (the logical intersection of two events, denoted ).
Contingency Table Example:
Probability that an individual eats junk food and watches sports on TV:
Marginal Probability Decomposition (Law of Total Probability):
The marginal probability of an event can be decomposed into the sum of its joint intersections with an entire set of disjoint partitions:
Structural constraints on events :
Mutually Exclusive: No two partitions can occur simultaneously ( for all ).
Collectively Exhaustive: One of the partition events must occur (the union covers the entire sample space: ).
Demographic Partition Example: Male and female categories represent mutually exclusive and collectively exhaustive groups.
Application to the Junk Food Survey:
The overall probability of eating junk food is partitioned across the two mutually exclusive television-watching states:
Verifying with numerical counts:
Rules of Probability: Addition and Event Types
Probability of One of Two Events Occurring ():
Refers to the logical union (), signifying that event happens, event happens, or both events happen.
Care must be taken to prevent double-counting outcomes that satisfy both conditions simultaneously.
Summation of Distinct Non-Overlapping Components:
Using individual disjoint cells from the contingency table to calculate :
Numerical evaluation:
The General Addition Rule:
The universal formula for the union of any two events:
Mechanism: Add the independent marginal probabilities of both events, then subtract the joint probability () to remove the duplicate intersection.
Numerical validation using marginal and joint figures:
Addition Rule for Mutually Exclusive Events:
If two events are mutually exclusive, they cannot co-occur; therefore, their joint probability is zero:
The addition formula reduces to:
Card Selection Demonstration:
When selecting a single card from a standard deck, determine :
A single card cannot be simultaneously a King and a Queen ().
Consequently:
Collectively Exhaustive Events:
A set of events is collectively exhaustive if it encompasses all possible outcomes of an experiment.
The sum of the probabilities across a collectively exhaustive set of mutually exclusive events always equals .
Examples:
Sex of a newborn baby:
Suits in a standard card deck:
Conditional Probability and Statistical Independence
Conditional Probability:
Conditional probability represents the likelihood of an event occurring given that a specific prior event has already occurred.
The conditional probability of event given that event has occurred is written as .
Formula:
Verbal description: The joint probability of and divided by the marginal probability of the conditioning event .
Contingency Table Evaluation:
Determine the probability that an individual does not watch sports, given that they eat junk food:
The prior knowledge that the person eats junk food reduces the probability of not watching sports from the overall unconditional rate of down to , showing that prior information influences the outcome.
Statistical Independence:
Two events are statistically independent if the occurrence of one event has no effect whatsoever on the probability of the other event.
Formal mathematical condition:
Equivalently, independence implies:
The Multiplication Rule:
General Multiplication Rule (derived directly by rearranging the conditional probability formula):
Multiplication Rule for Statistically Independent Events:
When , the joint probability formula becomes:
Sequential Sampling Demonstrations (With vs. Without Replacement):
Scenario 1: Dependent Selection (Sampling Without Replacement):
Two cards are drawn consecutively from a standard -card deck without replacing the first card.
Let be the event that the first card drawn is red ( red cards out of ).
Let be the event that the second card drawn is red ( red cards remaining out of total cards).
Calculation:
Scenario 2: Independent Selection (Sampling With Replacement):
Two cards are drawn consecutively, but the first card is placed back into the deck and the deck is reshuffled prior to the second draw.
Because the composition of the deck is restored, the selections are statistically independent:
Principles of Combinatorics: The Five Counting Rules
Need for Counting Rules:
Calculating probabilities for complex experiments requires systematically computing the total number of possible ways events can occur within a sample space.
Counting Rule 1: Independent Repeated Trials with Constant Outcome Spaces:
Applies when an experiment consists of independent trials, and any one of mutually exclusive and exhaustive outcomes can occur on each trial.
Total number of possible outcomes:
Counting Rule 2: Sequential Independent Trials with Differing Outcome Sets:
Applies to a sequence of trials where there are outcomes possible on trial 1, outcomes on trial 2, and outcomes on trial .
Total number of possible outcomes:
License Plate Example:
A license plate format composed of three letters followed by three numerical digits.
Assuming possible English letters for positions 1 through 3, and possible digits ( through ) for positions 4 through 6:
Counting Rule 3: Factorial Arrangements of Objects in Order:
Determines the total number of unique ways an entire set of distinct objects can be ordered or arranged sequentially.
Formula:
SEC East Standings Example:
Consider the athletic teams in the SEC East division.
Ignoring ties, the total number of possible final end-of-season standings arrangements is:
Counting Rule 4: Permutations of a Subset (Order Matters):
Computes the number of ways to select and arrange in specific order a subset of objects chosen from a total set of distinct objects.
Each ordered arrangement is termed a permutation.
Formula:
Class Leadership Selection Example:
A classroom contains students.
A committee of distinct ranked officer positions must be filled: President, Vice President (VP), and Secretary.
Because assigned roles matter, the sequence of selection produces different outcomes:
Counting Rule 5: Combinations (Order Does Not Matter):
Computes the number of ways to select a subset of objects from a group of total objects without regard to the order of selection.
Notation: Denoted by , , or standard binomial coefficient formatting:
Generic Committee Selection Example:
A classroom contains students, and a group of generic, unranked officers must be selected.
Because individual roles are not distinguished, order does not differentiate the resulting groups:
Bridge to Probability Distributions:
Combinations and the fundamental counting rules establish the mathematical framework required for analyzing discrete random variables and deriving the Binomial Probability Distribution.