1- Probability for Discrete Variables

Foundations and Fundamental Definitions of Probability

  • Core Concepts:

    • Experiment: An action, operation, or procedure that produces an event or a well-defined set of events.

    • Sample Space (SS): The comprehensive set comprising all possible elementary events or outcomes of an experiment. For discrete variables, outcomes can be nominal categories (e.g., X={blue,red,green}X = \{\text{blue}, \text{red}, \text{green}\}).

    • Event / Outcome (AA): A specific subset of outcomes within the sample space (e.g., drawing a blue ball from a container).

Illustration of sample space, experiment, and event using colored balls in an urn
  • Fundamental Probability Axioms / Rules:

    • Probability Range Rule: For any event AA occurring within sample space SS, the probability is bounded between zero and one inclusive:         0≤P(A)≤10 \le P(A) \le 1

    • Certainty of Sample Space: The probability that an outcome falls somewhere within the entire sample space equals one:         P(S)=1P(S) = 1

Marginal Probability and Complements

  • Marginal / Unconditional Probability:

    • Definition: The probability of a single discrete event occurring independently of any other events, denoted as P(A)P(A).

    • Equally Likely Elementary Outcomes: If a sample space consists of a finite total of NN equally likely elementary occurrences, and a specific discrete event AiA_i consists of NAiN_{A_i} of those occurrences, the marginal probability is calculated as:         P(Ai)=NAiNP(A_i) = \frac{N_{A_i}}{N}

  • Complement of an Event:

    • Definition: The complement of event AiA_i, denoted as Ai′A_i', is the event that AiA_i does not occur.

    • Formula: The probability of the complement is the remainder when the event's probability is subtracted from unity:         P(Ai′)=1−P(Ai)P(A_i') = 1 - P(A_i)

  • Worked Numerical Example (Urn/Bag Model):

    • A container holds N=20N = 20 total balls of nominal categories: blue, red, and green.

    • Number of blue balls: Nblue=6N_{\text{blue}} = 6.

    • Marginal probability of drawing a blue ball:         P(blue)=620=0.30P(\text{blue}) = \frac{6}{20} = 0.30

    • Complement probability of drawing a ball that is not blue:         P(blue′)=1−P(blue)=1−0.30=0.70P(\text{blue}') = 1 - P(\text{blue}) = 1 - 0.30 = 0.70

Set Operations and Event Relationships

Venn diagram showing intersection and complements
  • Intersection / Joint Probability (A∩BA \cap B):

    • Represents the simultaneous occurrence of both event AA and event BB.

    • Denoted as P(A∩B)P(A \cap B).

  • Set Complement Partitions:

    • AB′AB' represents all elements in the sample space that belong to event AA but do not belong to event BB.

    • BA′BA' represents all elements in the sample space that belong to event BB but do not belong to event AA.

  • Union of Events (A∪BA \cup B):

    • Represents the occurrence of either event AA, event BB, or both events simultaneously.

    • Denotes the occurrence of any event within the combined region:

      • The event AA alone is a possibility.

      • The event BB alone is a possibility.

      • The intersection A∩BA \cap B is a possibility.

  • Conditional Probability (A∣BA \mid B):

    • Represents the probability of event AA occurring given that event BB is known to have already occurred.

    • Denoted as P(A∣B)P(A \mid B).

  • Classification of Event Relationships:

    • Mutually Exclusive (Disjoint) Events:

      • Events AA and BB cannot occur at the same time.

      • The intersection contains zero elements: P(A∩B)=0P(A \cap B) = 0.

    • Independent Events:

      • The occurrence of event AA provides no information about the occurrence of event BB, and vice versa.

      • The conditional probability equals the unconditional marginal probability: P(A∣B)=P(A)P(A \mid B) = P(A).

    • Dependent Events:

      • The occurrence of event AA provides information regarding the occurrence of event BB (i.e., it alters the probability that BB occurs), and vice versa.

Sampling Strategies and Dependency

Sampling with and without replacement diagram
  • Sampling With Replacement:

    • After an item is selected from sample space SS, it is returned to the pool prior to the next draw.

    • The total number of items and the relative composition remain unchanged.

    • Successive draws remain strictly independent.

  • Sampling Without Replacement:

    • After an item is drawn from sample space SS, it is permanently removed and not returned to the pool.

    • In a finite sample space, removing elements shifts the denominator and relative proportions for subsequent draws.

    • Sampling without replacement directly creates dependencies among successive events.

Computational Probability Rules for Discrete Data

  • Additive Rule of Probability:

    • For Mutually Exclusive Events: When two discrete events AA and BB cannot occur together, the union equals the direct sum of their marginal probabilities:         P(A∪B)=P(A)+P(B)P(A \cup B) = P(A) + P(B)

    • General Additive Rule (Non-Mutually Exclusive Events): When two events can occur simultaneously, adding their marginal probabilities double-counts their overlap; the joint probability must be subtracted:         P(A∪B)=P(A)+P(B)−P(A∩B)P(A \cup B) = P(A) + P(B) - P(A \cap B)

  • Multiplicative Rule of Probability:

    • For Independent Events: The joint probability of two independent events equals the product of their individual marginal probabilities:         P(A∩B)=P(A)×P(B)P(A \cap B) = P(A) \times P(B)

    • For Dependent Events: The joint probability is computed using conditional probability:         P(A∩B)=P(A∣B)×P(B)P(A \cap B) = P(A \mid B) \times P(B)

  • Conditional Probability Formulation:

    • The formal definition of conditional probability is derived by rearranging the multiplicative rule:         P(A∣B)=P(A∩B)P(B)P(A \mid B) = \frac{P(A \cap B)}{P(B)}

  • Summary Matrix of Formulas across Event Types:

Probability Type

Mutually Exclusive Events

Independent Events

Dependent Events

Union: P(A∪B)P(A \cup B)

P(A∪B)=P(A)+P(B)P(A \cup B) = P(A) + P(B)

P(A∪B)=P(A)+P(B)−P(A∩B)P(A \cup B) = P(A) + P(B) - P(A \cap B)

P(A∪B)=P(A)+P(B)−P(A∩B)P(A \cup B) = P(A) + P(B) - P(A \cap B)

Joint: P(A∩B)P(A \cap B)

P(A∩B)=0P(A \cap B) = 0

P(A∩B)=P(A)×P(B)P(A \cap B) = P(A) \times P(B)

P(A∩B)=P(A∣B)×P(B)P(A \cap B) = P(A \mid B) \times P(B)

Conditional: P(A∣B)P(A \mid B)

P(A∣B)=0P(A \mid B) = 0

P(A∣B)=P(A)P(A \mid B) = P(A)

P(A∣B)=P(A∩B)P(B)P(A \mid B) = \frac{P(A \cap B)}{P(B)}

  • Deduction Requirements for Dependent Events:

    • To solve calculations involving dependent events, one must either be directly given the joint probability P(A∩B)P(A \cap B), given the conditional probability P(A∣B)P(A \mid B), or provided sufficient context to deduce them.

Summary table of probability formulas across event types

Discrete Probability Distributions

  • Probability Mass Function (PMF):

    • Definition: A function that maps each discrete outcome xix_i of a discrete random variable XX to its exact probability of occurrence, denoted as:         P(X=xi)for i=1,2,3,…,NP(X = x_i) \quad \text{for } i = 1, 2, 3, \dots, N

Probability mass function bar chart across Likert categories
  • Likert Category PMF Example:

    • A discrete PMF evaluated over five ordered categories (Likert_Category 1 through 5) exhibits the following empirical probability values:

      • Category 1: P(X=1)≈0.68P(X = 1) \approx 0.68

      • Category 2: P(X=2)≈0.27P(X = 2) \approx 0.27

      • Category 3: P(X=3)≈0.03P(X = 3) \approx 0.03

      • Category 4: P(X=4)≈0.02P(X = 4) \approx 0.02

      • Category 5: P(X=5)≈0.00P(X = 5) \approx 0.00

    • The sum of all discrete probability masses across the distribution equals one:         ∑i=15P(X=xi)=1.00\sum_{i=1}^{5} P(X = x_i) = 1.00

  • Joint Probability Mass Function (Contingency Structure):

    • For two independent binary events AA and BB and their respective complements A′A' and B′B', the joint PMF is organized in a cross-tabulated contingency table where inner cells represent intersections and outer margins represent marginal probabilities:

Event Partition

Event AA

Event A′A'

Marginal Row Probability

Event BB

P(A∩B)=P(A)×P(B)P(A \cap B) = P(A) \times P(B)

P(A′∩B)=P(A′)×P(B)P(A' \cap B) = P(A') \times P(B)

→P(B)\rightarrow P(B)

Event B′B'

P(A∩B′)=P(A)×P(B′)P(A \cap B') = P(A) \times P(B')

P(A′∩B′)=P(A′)×P(B′)P(A' \cap B') = P(A') \times P(B')

→P(B′)\rightarrow P(B')

Marginal Column Probability

→P(A)\rightarrow P(A)

→P(A′)\rightarrow P(A')

1.001.00

Joint probability mass function contingency table

Misinterpretations and Real-World Applications of Probability

  • Ecological Fallacy / Population vs. Individual Probability:

    • A reported group-level efficacy rate cannot be directly converted into an individual's personal probability.

    • Clinical Drug Example:

      • Scenario: A scientific study demonstrates that a novel pharmaceutical drug is effective in 90%90\% of study participants.

      • Fallacy: Assuming that any individual patient has a 90%90\% chance (P=0.90P = 0.90) of experiencing drug efficacy.

      • Reality: The chances that the drug was effective for a specific person (e.g., Participant 4 in the study) are unknown on an individual deterministic level (efficacy is an individual outcome, either 00 or 11).

      • Individual Inference: The exact probability that the drug will be effective for a new external individual is entirely unknown without specific individual conditional data.

  • Health and Lifetime Risk Reporting:

    • Epidemiological statistics are frequently presented as aggregate lifetime risks.

  • Risk Perception vs. Actuarial Reality:

    • Human risk perception commonly deviates from actuarial probability calculations, often magnifying rare, dramatic threats while underestimating commonplace hazards.