Random Sampling, Probability Theory, and Combinatorial Counting Rules

Foundations of Random Sampling

  • Definition of a Simple Random Sample:

    • A simple random sample of nn items from a population requires the satisfaction of two core conditions:

      • Condition (a): Every member of the population possesses the exact same probability of being included in the sample.

      • Condition (b): The members of the sample are selected completely independently of one another.

  • Methods and Tools for Generating Random Samples:

    • Obtaining an authentic random sample in practice can present significant operational challenges.

    • Standard tools and methods utilized to generate random samples include:

      • Calculators with built-in pseudorandom generator functions.

      • Computer algorithms designed for automated random selection.

      • Research Randomizer (an online utility designed for instant random sampling and random assignment).

      • Standardized random number tables.

  • Purpose of Random Sampling in Statistical Inference:

    • In order to draw valid inferences about a broader population from an observed sample, the sample must provide an accurate and unbiased representation of the population.

    • Implementing a formal random sampling model is the most effective approach to ensure representative sampling.

    • Once a representative random sample is established, researchers can validly apply the rules of probability to draw mathematical inferences about the target population.

Probability Definitions and Frameworks

  • Fundamental Bounds and Definition of Probability:

    • Probabilities are real numerical values defined on the closed interval between 00 and 11, inclusive (0≤P(A)≤10 \le P(A) \le 1), that quantify the likelihood of a specific event occurring.

    • Probabilities approaching 11 signify that the event is highly likely to take place.

    • Probabilities approaching 00 signify that the event is highly unlikely to take place.

    • A probability value of precisely 11 denotes certainty: the event will occur.

    • A probability value of precisely 00 denotes impossibility: the event will not occur.

  • Equally Likely Outcomes Model:

    • When an experiment has a finite number of possible outcomes that are all equally likely to occur, the probability of an event AA is formally defined as:         P(A)=XTP(A) = \frac{X}{T}

    • In this formulation:

      • XX represents the total number of favorable outcomes for which the event AA occurs.

      • TT represents the total number of all possible outcomes within the sample space.

  • Simple Probability:

    • Simple probability denotes the likelihood of occurrence of a single, individual simple event, denoted symbolically as P(A)P(A).

  • Three Approaches to Determining Probability:

    • Determining probability varies depending on the specific application, environment, and constraints under study:

      • Subjective Probability: Probabilities based on personal judgment, qualitative assessment, beliefs, or subjective expectations rather than mathematical proofs or frequency data.

      • A Priori Classical Probability: Probabilities derived deductively from prior knowledge and theoretical understanding of the physical or mathematical process generating the outcomes.

      • Empirical Classical Probability: Probabilities established inductively from observed relative frequencies within empirical sample data or experimental trials.

Classical, Empirical, and Subjective Probability Applications

  • A Priori Classical Probability (Marginal Probability):

    • Computed purely from structural understanding of the system:         P(A)=number of ways that the event of interest can happennumber of total possible eventsP(A) = \frac{\text{number of ways that the event of interest can happen}}{\text{number of total possible events}}

    • This measure is also referred to as marginal probability when focused on a single categorical dimension.

    • Standard Playing Card Deck Application:

      • Objective: Determine the probability of randomly drawing a face card from a standard deck of 5252 playing cards.

      • A standard deck contains 1212 face cards (four Jacks, four Queens, and four Kings) out of 5252 total cards.

      • Calculation:             P(Face Card)=1252≈0.23P(\text{Face Card}) = \frac{12}{52} \approx 0.23

  • Empirical Classical Probability via Contingency Tables:

    • Empirical probability is calculated from observed counts structured in a two-way contingency table.

    • Survey Data Contingency Table (N=200N = 200 individuals):

        | Category | Eats Junk Food | Does Not Eat Junk Food | Total |         | :--- | :--- | :--- | :--- |         | Does not Watch Sports on TV | 4545 | 5050 | 9595 |         | Watches Sports on TV | 8585 | 2020 | 105105 |         | Total | 130130 | 7070 | 200200 |

*   *Empirical Calculations from Table*:
    *   Probability that a randomly selected individual does not watch sports on TV:

            P(Does not watch sports)=95200=0.475P(\text{Does not watch sports}) = \frac{95}{200} = 0.475 * Probability that a randomly selected individual eats junk food:             P(Eats junk food)=130200=0.65P(\text{Eats junk food}) = \frac{130}{200} = 0.65             (Note: Listed in raw source text as 130/300=0.65130/300 = 0.65, which contains a typographical denominator for the grand total sample size of 200200, yielding the value 0.650.65.)

  • Real-World Context: Gambling Paradigms:

    • Different forms of gambling correspond directly to the three probability processes:

      • Subjective Probability: Governs wagering on competitive sports such as horse racing and football games, where external factors, expert consensus, and fluctuating conditions dictate the odds.

      • Empirical Classical Probability: Governs strategic games such as poker to a certain extent, where players calculate frequencies based on observed play styles and empirical card distributions.

      • A Priori Classical Probability: Governs mechanical casino games such as rolling dice and roulette wheels, where the sample space and exact probabilities are dictated entirely by geometry and physics.

Joint Probability and Marginal Decompositions

  • Joint Probability:

    • Joint probability measures the chance that two distinct events take place concurrently (the logical intersection of two events, denoted P(A and B)P(A \text{ and } B)).

    • Contingency Table Example:

      • Probability that an individual eats junk food and watches sports on TV:             P(Junk food and Watches sports)=85200=0.425P(\text{Junk food and Watches sports}) = \frac{85}{200} = 0.425

  • Marginal Probability Decomposition (Law of Total Probability):

    • The marginal probability of an event AA can be decomposed into the sum of its joint intersections with an entire set of disjoint partitions:         P(A)=P(A and B1)+P(A and B2)+⋯+P(A and Bk)P(A) = P(A \text{ and } B_1) + P(A \text{ and } B_2) + \dots + P(A \text{ and } B_k)

    • Structural constraints on events B1,B2,…,BkB_1, B_2, \dots, B_k:

      • Mutually Exclusive: No two partitions can occur simultaneously (Bi∩Bj=∅B_i \cap B_j = \emptyset for all i≠ji \ne j).

    • Collectively Exhaustive: One of the partition events must occur (the union covers the entire sample space: ⋃i=1kBi=S\bigcup_{i=1}^k B_i = S).

    • Demographic Partition Example: Male and female categories represent mutually exclusive and collectively exhaustive groups.

  • Application to the Junk Food Survey:

    • The overall probability of eating junk food is partitioned across the two mutually exclusive television-watching states:         P(Junk food)=P(Junk food and No sports)+P(Junk food and Sports)P(\text{Junk food}) = P(\text{Junk food and No sports}) + P(\text{Junk food and Sports})

    • Verifying with numerical counts:         P(Junk food)=45200+85200=130200=0.65P(\text{Junk food}) = \frac{45}{200} + \frac{85}{200} = \frac{130}{200} = 0.65

Rules of Probability: Addition and Event Types

  • Probability of One of Two Events Occurring (A or BA \text{ or } B):

    • Refers to the logical union (A∪BA \cup B), signifying that event AA happens, event BB happens, or both events happen.

    • Care must be taken to prevent double-counting outcomes that satisfy both conditions simultaneously.

  • Summation of Distinct Non-Overlapping Components:

    • Using individual disjoint cells from the contingency table to calculate P(Eats junk food or Watches sports)P(\text{Eats junk food or Watches sports}):         P(Eats junk food or Watches sports)=P(No sports and Junk food)+P(Watches sports and Not junk food)+P(Watches sports and Junk food)P(\text{Eats junk food or Watches sports}) = P(\text{No sports and Junk food}) + P(\text{Watches sports and Not junk food}) + P(\text{Watches sports and Junk food})

    • Numerical evaluation:         P(Eats junk food or Watches sports)=45200+20200+85200=150200=0.75P(\text{Eats junk food or Watches sports}) = \frac{45}{200} + \frac{20}{200} + \frac{85}{200} = \frac{150}{200} = 0.75

  • The General Addition Rule:

    • The universal formula for the union of any two events:         P(A or B)=P(A)+P(B)−P(A and B)P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)

    • Mechanism: Add the independent marginal probabilities of both events, then subtract the joint probability (P(A and B)P(A \text{ and } B)) to remove the duplicate intersection.

    • Numerical validation using marginal and joint figures:         P(Junk food or Sports)=130200+105200−85200=150200=0.75P(\text{Junk food or Sports}) = \frac{130}{200} + \frac{105}{200} - \frac{85}{200} = \frac{150}{200} = 0.75

  • Addition Rule for Mutually Exclusive Events:

    • If two events are mutually exclusive, they cannot co-occur; therefore, their joint probability is zero:         P(A and B)=0P(A \text{ and } B) = 0

    • The addition formula reduces to:         P(A or B)=P(A)+P(B)P(A \text{ or } B) = P(A) + P(B)

    • Card Selection Demonstration:

      • When selecting a single card from a standard deck, determine P(Queen or King)P(\text{Queen or King}):             P(Queen or King)=P(Queen)+P(King)−P(King and Queen)P(\text{Queen or King}) = P(\text{Queen}) + P(\text{King}) - P(\text{King and Queen})

      • A single card cannot be simultaneously a King and a Queen (P(King and Queen)=0P(\text{King and Queen}) = 0).

      • Consequently:             P(Queen or King)=P(Queen)+P(King)P(\text{Queen or King}) = P(\text{Queen}) + P(\text{King})

  • Collectively Exhaustive Events:

    • A set of events is collectively exhaustive if it encompasses all possible outcomes of an experiment.

    • The sum of the probabilities across a collectively exhaustive set of mutually exclusive events always equals 11.

    • Examples:

      • Sex of a newborn baby:             P(Boy)+P(Girl)=1P(\text{Boy}) + P(\text{Girl}) = 1

      • Suits in a standard card deck:             P(Spade)+P(Heart)+P(Club)+P(Diamond)=1P(\text{Spade}) + P(\text{Heart}) + P(\text{Club}) + P(\text{Diamond}) = 1

Conditional Probability and Statistical Independence

  • Conditional Probability:

    • Conditional probability represents the likelihood of an event occurring given that a specific prior event has already occurred.

    • The conditional probability of event AA given that event BB has occurred is written as P(A∣B)P(A|B).

    • Formula:         P(A∣B)=P(A and B)P(B)P(A|B) = \frac{P(A \text{ and } B)}{P(B)}

    • Verbal description: The joint probability of AA and BB divided by the marginal probability of the conditioning event BB.

    • Contingency Table Evaluation:

      • Determine the probability that an individual does not watch sports, given that they eat junk food:             P(No sports∣Junk food)=P(No sports and Junk food)P(Junk food)=45200130200=45130≈0.346P(\text{No sports} | \text{Junk food}) = \frac{P(\text{No sports and Junk food})}{P(\text{Junk food})} = \frac{\frac{45}{200}}{\frac{130}{200}} = \frac{45}{130} \approx 0.346

      • The prior knowledge that the person eats junk food reduces the probability of not watching sports from the overall unconditional rate of 0.4750.475 down to 0.3460.346, showing that prior information influences the outcome.

  • Statistical Independence:

    • Two events are statistically independent if the occurrence of one event has no effect whatsoever on the probability of the other event.

    • Formal mathematical condition:         P(A∣B)=P(A)P(A|B) = P(A)

    • Equivalently, independence implies:         P(B∣A)=P(B)P(B|A) = P(B)

  • The Multiplication Rule:

    • General Multiplication Rule (derived directly by rearranging the conditional probability formula):         P(A and B)=P(A∣B)×P(B)P(A \text{ and } B) = P(A|B) \times P(B)

    • Multiplication Rule for Statistically Independent Events:

      • When P(A∣B)=P(A)P(A|B) = P(A), the joint probability formula becomes:             P(A and B)=P(A)×P(B)P(A \text{ and } B) = P(A) \times P(B)

  • Sequential Sampling Demonstrations (With vs. Without Replacement):

    • Scenario 1: Dependent Selection (Sampling Without Replacement):

      • Two cards are drawn consecutively from a standard 5252-card deck without replacing the first card.

      • Let BB be the event that the first card drawn is red (2626 red cards out of 5252).

      • Let AA be the event that the second card drawn is red (2525 red cards remaining out of 5151 total cards).

      • Calculation:             P(A red and B red)=P(A red∣B red)×P(B red)=(2551)×(2652)P(A \text{ red and } B \text{ red}) = P(A \text{ red} | B \text{ red}) \times P(B \text{ red}) = \left(\frac{25}{51}\right) \times \left(\frac{26}{52}\right)

    • Scenario 2: Independent Selection (Sampling With Replacement):

      • Two cards are drawn consecutively, but the first card is placed back into the deck and the deck is reshuffled prior to the second draw.

      • Because the composition of the deck is restored, the selections are statistically independent:             P(A red and B red)=P(A red)×P(B red)=(2652)×(2652)P(A \text{ red and } B \text{ red}) = P(A \text{ red}) \times P(B \text{ red}) = \left(\frac{26}{52}\right) \times \left(\frac{26}{52}\right)

Principles of Combinatorics: The Five Counting Rules

  • Need for Counting Rules:

    • Calculating probabilities for complex experiments requires systematically computing the total number of possible ways events can occur within a sample space.

  • Counting Rule 1: Independent Repeated Trials with Constant Outcome Spaces:

    • Applies when an experiment consists of nn independent trials, and any one of kk mutually exclusive and exhaustive outcomes can occur on each trial.

    • Total number of possible outcomes:         Total Outcomes=kn\text{Total Outcomes} = k^n

  • Counting Rule 2: Sequential Independent Trials with Differing Outcome Sets:

    • Applies to a sequence of nn trials where there are k1k_1 outcomes possible on trial 1, k2k_2 outcomes on trial 2, and knk_n outcomes on trial nn.

    • Total number of possible outcomes:         Total Outcomes=k1×k2×⋯×kn\text{Total Outcomes} = k_1 \times k_2 \times \dots \times k_n

    • License Plate Example:

      • A license plate format composed of three letters followed by three numerical digits.

      • Assuming 2626 possible English letters for positions 1 through 3, and 1010 possible digits (00 through 99) for positions 4 through 6:             Total Plates=26×26×26×10×10×10=263×103=17,576,000\text{Total Plates} = 26 \times 26 \times 26 \times 10 \times 10 \times 10 = 26^3 \times 10^3 = 17,576,000

  • Counting Rule 3: Factorial Arrangements of nn Objects in Order:

    • Determines the total number of unique ways an entire set of nn distinct objects can be ordered or arranged sequentially.

    • Formula:         n!=n(n−1)(n−2)…(1)n! = n(n - 1)(n - 2) \dots (1)

    • SEC East Standings Example:

      • Consider the 66 athletic teams in the SEC East division.

      • Ignoring ties, the total number of possible final end-of-season standings arrangements is:             6!=6×5×4×3×2×1=7206! = 6 \times 5 \times 4 \times 3 \times 2 \times 1 = 720

  • Counting Rule 4: Permutations of a Subset (Order Matters):

    • Computes the number of ways to select and arrange in specific order a subset of XX objects chosen from a total set of nn distinct objects.

    • Each ordered arrangement is termed a permutation.

    • Formula:         nPX=n!(n−X)!_nP_X = \frac{n!}{(n - X)!}

    • Class Leadership Selection Example:

      • A classroom contains n=10n = 10 students.

      • A committee of X=3X = 3 distinct ranked officer positions must be filled: President, Vice President (VP), and Secretary.

      • Because assigned roles matter, the sequence of selection produces different outcomes:             10P3=10!(10−3)!=10!7!=10×9×8=720 possible officer configurations_{10}P_3 = \frac{10!}{(10 - 3)!} = \frac{10!}{7!} = 10 \times 9 \times 8 = 720\text{ possible officer configurations}

  • Counting Rule 5: Combinations (Order Does Not Matter):

    • Computes the number of ways to select a subset of XX objects from a group of nn total objects without regard to the order of selection.

    • Notation: Denoted by CC, nCX_nC_X, or standard binomial coefficient formatting:         (nX)=n!X!(n−X)!\binom{n}{X} = \frac{n!}{X!(n - X)!}

    • Generic Committee Selection Example:

      • A classroom contains n=10n = 10 students, and a group of X=3X = 3 generic, unranked officers must be selected.

      • Because individual roles are not distinguished, order does not differentiate the resulting groups:             (103)=10!3!(10−3)!=10!3!×7!=10×9×83×2×1=7206=120 possible combinations\binom{10}{3} = \frac{10!}{3!(10 - 3)!} = \frac{10!}{3! \times 7!} = \frac{10 \times 9 \times 8}{3 \times 2 \times 1} = \frac{720}{6} = 120\text{ possible combinations}

  • Bridge to Probability Distributions:

    • Combinations and the fundamental counting rules establish the mathematical framework required for analyzing discrete random variables and deriving the Binomial Probability Distribution.