Comprehensive Study Notes on Probability, Discrete Distributions, and Normal Distributions
Foundations of Probability and Random Processes
Definition of a Random Process:
- A random process represents scenarios where the outcome of any particular trial of an experiment is unknown, but the proportion (or relative frequency) with which a particular outcome is observed approaches a specific, constant value as the number of trials increases.
- Probability is the measure of the likelihood of a random phenomenon or chance behavior occurring.
- Probability deals with experiments that yield unpredictable, random short-term results, yet reveal structured long-term predictability.
- The long-term proportion in which a certain outcome is observed defines the probability of that outcome.
The Law of Large Numbers (LLN):
- As the number of repetitions of a probability experiment increases, the proportion with which a certain outcome is observed gets closer to the theoretical probability of that outcome.
- The Nonexistent "Law of Averages": The Law of Large Numbers describes long-term behavior over many trials; it does not apply to short-term sequences. A common misconception (the gambler's fallacy) posits that an outcome is "due" because it has not occurred recently (e.g., expecting heads after rolling or flipping five tails in a row, or asserting a baseball batter is due for a hit after several hitless at-bats).
- Trials in random processes are memoryless: the mechanism behind the process does not recall past results, and the likelihood of observing any outcome on the next trial remains identical to its likelihood on the first trial.
Probability Experiments, Sample Spaces, and Events:
- Experiment: Any process that can be repeated and produces uncertain outcomes.
- Sample Space (): The collection of all possible outcomes of a probability experiment.
- Event: Any collection of outcomes from a probability experiment. An event may consist of a single outcome or multiple outcomes. Events are typically denoted by capital letters (such as ).
- Simple Event (): An event containing exactly one outcome.
- Example (Rock, Paper, Scissors played twice):
- Outcomes represent ordered pairs of selections across the two rounds.
- Sample space: .
- Event : .
The Rules of Probabilities:
- Rule 1 (Probability Boundaries): For any event , the probability must be greater than or equal to and less than or equal to :
- Rule 2 (Sum of Probabilities): The sum of the probabilities of all simple outcomes in the sample space must equal :
- Probability Model: A comprehensive listing of all possible outcomes of a probability experiment along with their associated probabilities. To be valid, a probability model must satisfy both Rule 1 and Rule 2.
- Certainty and Impossibility:
- If an event cannot happen (impossible), .
- If an event is guaranteed to happen (certainty), .
- An event with probability is expected to occur approximately times out of every repetitions of the experiment.
- Unusual Event: An event that has a low probability of occurring. The conventional threshold for classifying an event as unusual is a probability strictly less than or equal to ().
Methods of Computing Probabilities:
- The Empirical Approach (Relative Frequency): Approximates the probability of an event based on observed experimental data:
- The Classical Approach (Theoretical): Applicable when an experiment has equally likely outcomes and event can occur in different ways: where is the number of outcomes in event and is the total number of outcomes in the sample space .
- Subjective Probability: A probability value assigned on the basis of personal judgment, expertise, or individual belief rather than relative frequencies or counting equally likely outcomes (e.g., an economic forecast predicting a chance of recession, or racetrack betting odds set by betting volume).
Empirical Probability Case Study (Pass the Pigs):
- In a game of Pass the Pigs (Milton-Bradley), a class of students tossed a pig die a total of times, recording six landing positions:
- Side with no dot: frequency ()
- Side with dot: frequency ()
- Razorback: frequency ()
- Trotter: frequency ()
- Snouter: frequency ()
- Leaning Jowler: frequency ()
- Because , obtaining a Leaning Jowler is classified as an unusual event.
- In a game of Pass the Pigs (Milton-Bradley), a class of students tossed a pig die a total of times, recording six landing positions:
Classical Probability Case Study (Peanut and Fun Size M&M Bags):
- Peanut M&M Distribution Model: Brown (), Yellow (), Red (), Blue (), Orange (), Green (). All probabilities satisfy and sum exactly to , confirming validity.
- Fun Size Bag Analysis: A bag contains brown, yellow, red, orange, blue, and green candies ( total candies): Drawing a yellow candy is three times more likely than drawing a blue candy.
Probability Rules for Combined Events and Complements
Disjoint (Mutually Exclusive) Events:
- Two events are disjoint (or mutually exclusive) if they have no outcomes in common. That is, both events cannot occur simultaneously on a single trial.
- Addition Rule for Disjoint Events: If and are disjoint events, then:
- Generalization to Multiple Disjoint Events: If events are pairwise disjoint, then:
- Example (Housing Units in the United States):
- One room:
- Two rooms:
- Three rooms:
- Four rooms:
- Five rooms:
- Six rooms:
- Seven rooms:
- Eight rooms:
- Nine or more rooms:
The General Addition Rule:
- When two events and are not disjoint, adding their individual probabilities double-counts the outcomes in their intersection ().
- Formula: For any two events and :
- Example: If , , and :
- If and were mutually exclusive instead, , yielding .
Contingency Tables and Probability Calculations:
- Contingency tables record joint frequencies for two categorical variables across a defined population or sample.
- Example (2021 American Community Survey Living Arrangements, in millions of households; Total = ):
- 1-person household: Owner-Occupied = , Renter-Occupied = (Subtotal = )
- 2-person household: Owner-Occupied = , Renter-Occupied = (Subtotal = )
- 3-person household: Owner-Occupied = , Renter-Occupied = (Subtotal = )
- 4-or-more-person household: Owner-Occupied = , Renter-Occupied = (Subtotal = )
- Totals: Total Owner-Occupied = , Total Renter-Occupied = .
- Calculations:
The Complement Rule:
- Complement of an Event (): The set of all outcomes in the sample space that are not contained in event .
- Formula: For any event :
- Examples:
- If the American Veterinary Medical Association reports that () of American households own a dog, the probability that a randomly chosen household does not own a dog is:
- Commute time in Hartford County, CT: If , then the probability of commuting less than minutes is .
Independence, Conditional Probability, and Multiplication Rules
Independent versus Dependent Events:
- Independent Events: Two events and are independent if the occurrence of event does not alter the probability that event occurs.
- Dependent Events: Two events are dependent if the occurrence of event affects the probability of event .
- Disjoint versus Independent: Disjointness and independence are fundamentally distinct concepts. If two events with non-zero probability are disjoint, they cannot be independent, because knowing that event occurred guarantees that event cannot occur ().
The Multiplication Rule for Independent Events:
- If and are independent events, then:
- Extension to Independent Events: If are independent events:
- Applications:
- If () of residents hold a bachelor's degree or higher: for two randomly selected individuals, . For four individuals, . For three holding a degree and the fourth not: .
- Exercise Equipment Warranty: A manufacturer knows of equipment is defective () and of customers use it in the first year (). Assuming independence, the proportion making a valid claim within the 1-year warranty is:
Computing "At-Least" Probabilities:
- The phrase "at least one" represents the complement of "none".
- Formula:
- Example: Among randomly selected residents where hold a degree:
Conditional Probability:
- The notation represents the probability of event occurring given that event has already occurred.
- Formula:
- Pet Ownership Survey (Algebra Students):
- Male students: Dog , Cat , Bird (Total = )
- Female students: Dog , Cat , Bird (Total = )
- Overall totals: Dog , Cat , Bird ; Grand total =
- Household Connectivity Survey (Sample of Households):
- High-speed internet ():
- Land-line phone service ():
- Mobile phone service ():
- High-speed and land-line ():
- Land-line and mobile ():
- Of those with mobile service, have high-speed ()
The General Multiplication Rule:
- Derived directly from rearranging the conditional probability formula:
- Equivalence Condition for Independence: Two events and are independent if and only if:
Counting Techniques and Combined Probability Problems
The Multiplication Rule of Counting:
- If a task consists of a sequence of choices where there are selections for the first choice, selections for the second choice, selections for the third choice, and so on, the total number of distinct ways to complete the task is:
- Fixed-Price Dinner Menu Example (Merichka's Restaurant):
- Appetizer ( options): soup or salad
- Entrée ( options): baked chicken, broiled beef patty, baby beef liver, or roast beef au jus
- Dessert ( options): ice cream or cheesecake
- Total unique meal selections:
Factorial Notation:
- For any non-negative integer :
- By definition: and .
- Arranging Test Questions: If an instructor selects questions from a test bank of , the number of distinct ways to arrange those chosen questions on an exam is .
Permutations of Distinct Objects:
- A permutation is an ordered arrangement of objects selected from distinct objects without replacement (), where order is significant.
- Formula:
- Trifecta Wager Example: In a -horse race, the number of ways horses can finish in first, second, and third place is:
Combinations of Distinct Objects:
- A combination is an arrangement of objects chosen from distinct objects without replacement (), where order does not matter.
- Formula:
- Carrot Treat Example: The number of ways to select horses from a group of to receive a carrot treat is:
Permutations with Nondistinct Items:
- The number of distinct permutations of objects of which are of one type, are of a second type, …, and are of a th type (where ) is given by:
- Arranging Letters in "MISSISSIPPI":
- Total letters:
- Letter counts: , , ,
- Distinct arrangements:
Applications of Counting to Probability:
- Guiseppe's Pizza: Deal offering a large pizza with available toppings (sausage, pepperoni, mushroom, olive, tomato, spinach, onion, extra cheese).
- Total possible two-topping pizzas: .
- Two-topping pizzas containing pepperoni: .
- Probability of getting pepperoni: .
- Friday Night Free Pizza Drawing: Out of orders, are randomly chosen to be free. If an individual orders pizzas:
- Total ways to choose winning pizzas from : .
- Ways to receive zero free pizzas: .
- Probability of receiving at least one free pizza: .
- Marbles in a Jar Problem: A jar contains red, blue, and yellow marbles (total ). Three marbles are selected at random without replacement:
- Total selections: .
- Selections with exactly two red and one blue: .
- Probability: .
- Child Advocacy Center Case Distribution (Total = Cases):
- Age under 1: cases (; unusual since )
- Age 1 to 5: cases ()
- Age 6 to 10: cases ()
- Age 11 to 14: cases ()
- Age 15 to 17: cases ()
- Guiseppe's Pizza: Deal offering a large pizza with available toppings (sausage, pepperoni, mushroom, olive, tomato, spinach, onion, extra cheese).
Discrete Random Variables and Probability Distributions
- Distinguishing Discrete and Continuous Random Variables:
- Random Variable (): A numerical measure of the outcome of a probability experiment, whose value is determined by chance.
- Discrete Random Variable: Possesses either a finite or a countable number of values. Its values can be plotted as discrete points on a number line with gaps separating them.
- Continuous Random Variable: Possesses infinitely many values corresponding to intervals on a real line, plotted in an uninterrupted manner without gaps.

* *Classification Examples (16-ounce Bag of M&Ms):*
* Mass of each individual M&M: Continuous (measured weight, e.g., ).
* Number of pieces of candy in the bag: Discrete (counted values, ).
* Diameter of each piece of candy: Continuous (measured dimension, e.g., ).
* Number of distinct colors in the bag: Discrete (counted values, ).
Rules for a Discrete Probability Distribution:
- A discrete probability distribution lists all possible numerical values that the random variable can assume, along with the corresponding probability .
- Requirements:
- for all individual values of
Mean, Expected Value, and Standard Deviation of a Discrete Random Variable:
- Mean (Expected Value, ): Represents the theoretical weighted average of the outcomes in the long run:
- Interpretation of the Mean: If an experiment is repeated independent times and the values of are recorded as , the sample mean: approaches as increases, causing the difference to converge to .
- Standard Deviation: Measures the dispersion of the distribution around its mean:
Case Study: Chuck-a-Luck Game Analysis:
- Game Mechanics: Three dice are tumbled in a wire cage. A player wagers 1\ on a selected integer from to . If the chosen number appears on dice (), the player receives k\ in winnings and gets their 1\ stake back. If no dice match (), the player loses their 1\ stake.
- Distribution Table:
- matches (x = -1\):
- match (x = +1\):
- matches (x = +2\):
- matches (x = +3\):
- Verification: .
- Mean Computation:Interpretation: In the long run, a player expects to lose an average of approximately cents (or cents) per 1\ wager. In a simulated trial of games, the running mean converges directly toward this theoretical average (-0.08\).
- Standard Deviation Computation:
- \sigma_X = \sqrt{1.2453 - (-0.0787)^2} = \sqrt{1.2453 - 0.00619} = \sqrt{1.2391} \approx 1.1131\
Case Study: Expected Profit on Real Estate Renovation:
- Shawn and Maddie acquire a foreclosed property for 50,000\ and spend an additional 27,000\ on repairs (total fixed cost = 77,000\).
- Selling Price Scenarios and Net Profit ():
- Resell at 120,000\ (x = 43,000\),
- Resell at 100,000\ (x = 23,000\),
- Resell at 80,000\ (x = 3,000\),
- Resell at 60,000\ (x = -17,000\),
- Expected Net Profit Computation: E(X) = 6,450 + 10,350 + 750 - 2,550 = 15,000\ Interpretation: The expected average net profit across many similar real estate ventures is 15,000\.
The Binomial Probability Distribution
Criteria for a Binomial Probability Experiment:
- The experiment is conducted for a fixed number of trials, denoted by .
- The trials are independent; the outcome of one trial has no influence on the outcomes of any other trials.
- Each trial has exactly two mutually exclusive (disjoint) outcomes: success or failure.
- The probability of success, denoted by , remains constant for each trial. The probability of failure is .
Binomial Notation and Probability Distribution Function:
- : Total number of independent trials.
- : Constant probability of success on any single trial.
- : Probability of failure on any single trial.
- : Discrete random variable denoting the total number of successes in trials ().
- Binomial Probability Formula:
Mean and Standard Deviation of a Binomial Distribution:
- Mean (Expected Number of Successes):
- Standard Deviation:
Shape and Approximate Normality of the Binomial Distribution:
- For a fixed value of , as the number of trials increases, the binomial probability distribution becomes progressively more symmetric and bell-shaped.
- Rule of Thumb for Bell-Shaped Distribution: When this condition is satisfied, the distribution is approximately normal.
Worked Examples (Binomial Distribution):
- McCarran International Airport Flight Punctuality:
- Parameters: In April 2023, on-time proportion was . A sample of flights is randomly selected.
- Sample space values for : .
- Public Opinion on Divorce (Harris International):
- Parameters: Proportion of adult Americans accepting divorce is . Sample size .
- Exactly 20 accept:
- Fewer than 10 accept:
- At least 10 accept:
- Between 18 and 22 accept, inclusive:
- Evaluating Unusual Observations (, ):
- Mean:
- Standard deviation:
- Normality condition: (well satisfied).
- Usual interval (Empirical Rule, ): .
- Result: An outcome of successes falls within this interval (); therefore, observing people who believe divorce is acceptable is not unusual.
- McCarran International Airport Flight Punctuality:
The Poisson Probability Distribution
Criteria for a Poisson Process:
- A discrete random variable , representing the number of occurrences (successes) of an event over a fixed interval of time or space, follows a Poisson process if:
- The probability of two or more successes in any sufficiently small subinterval is .
- The probability of success is identical for any two subintervals of equal length.
- The number of successes in any interval is independent of the number of successes in any other interval, provided the intervals do not overlap.
- A discrete random variable , representing the number of occurrences (successes) of an event over a fixed interval of time or space, follows a Poisson process if:
Poisson Probability Distribution Function:
- If represents the number of occurrences in an interval of fixed length , and occurrences arrive at an average rate of events per unit interval of length , the probability of observing exactly occurrences is: where .
Mean and Standard Deviation of a Poisson Random Variable:
- Mean (Expected Value):
- Standard Deviation:
Worked Examples (Poisson Distribution):
- Textbook Typos Analysis:
- Parameters: Typos occur at a rate of per page. The interval is pages. Thus, expected occurrences: .
- Probability of exactly 10 typos:
- Probability of less than 10 typos:
- Probability of at least 10 typos:
- Hospital Emergency Room Arrivals:
- Parameters: Patients arrive at a rate of arrivals per minute between 9:00 p.m. and midnight. Interval length .
- Expected arrivals per interval: .
- Standard deviation: .
- Probability of exactly 10 arrivals:
- Unusual Threshold Evaluation (more than 20 arrivals):
- Usual interval: .
- Because falls within two standard deviations (), observing more than arrivals is not unusual according to the two-standard-deviation criterion, although can be calculated precisely using technology to confirm whether .
- Textbook Typos Analysis:
Properties and Applications of the Normal Distribution
Continuous Uniform Distribution:
- A continuous random variable has a uniform distribution over an interval if the likelihood of landing in any subinterval depends exclusively on the subinterval's length.
- Amazon Delivery Example: Delivery is scheduled between 10:00 a.m. and 11:00 a.m. Let be minutes elapsed past 10:00 a.m. (). The density function is constant: for .
- Probability Calculation: The probability that the package arrives between 10:15 a.m. and 10:30 a.m. () corresponds to the rectangular area:
- To find the time within which there is a probability of arrival: .
Properties of Probability Density Functions (pdf):
- A probability density function for a continuous random variable must satisfy:
- The total area under the graph over all possible values of the random variable equals .
- The curve lies on or above the horizontal axis everywhere: for all possible values of .
- The area under the graph of a density function between two values and equals the probability that the random variable assumes a value in that interval: .
- For any continuous random variable, the probability of assuming any exact single value is zero (); therefore, .
- A probability density function for a continuous random variable must satisfy:
Properties of the Normal Density Curve:
- The normal curve is symmetric about its population mean, .
- Because , the curve exhibits a single peak occurring precisely at .
- The curve has inflection points (points where concavity changes) at and .
- The total area bounded under the normal curve and the horizontal axis is exactly .
- Due to symmetry, the area under the curve to the left of is (), and the area to the right of is ().
- As increases or decreases without bound ( or ), the graph approaches, but never touches or crosses, the horizontal axis (the horizontal axis serves as an asymptote).
- The Empirical Rule: For any normally distributed random variable:
- Approximately of the area lies within standard deviation of the mean: .
- Approximately of the area lies within standard deviations of the mean: .
- Approximately of the area lies within standard deviations of the mean: .

Role of Parameters in the Normal Curve:
- Changing the mean shifts the normal curve along the horizontal axis without modifying its spread or shape.
- Changing the standard deviation alters the spread of the distribution: larger values of flatten and widen the curve, whereas smaller values compress and heighten the curve around .
Interpreting Area under the Normal Curve:
- If is normally distributed with parameters and , the area under the curve over an interval represents:
- The proportion of the population that possesses the characteristic defined by the interval.
- The probability that a randomly chosen individual from the population possesses that characteristic.
- Major League Baseball (MLB) Home Run Example (2022 Season):
- Distance traveled is normally distributed with and .
- If the area under the curve to the right of is :
- The proportion of all home runs hit during the 2022 season that traveled more than is ().
- The probability that a randomly chosen home run traveled more than is .
- If is normally distributed with parameters and , the area under the curve over an interval represents:
Standardizing Values Using -Scores:
- The standard normal distribution has mean and standard deviation . Any normal variable can be converted to a standard normal variable via:
- Reporting Extreme Tail Probabilities: Because the normal curve extends infinitely in both directions, the cumulative area to the left or right of any finite value cannot mathematically equal or . When computational tools round extreme cumulative areas to or , they should be recorded as or , respectively.
Worked Application Examples (Normal Distribution):
- Ball Bearing Factory Specifications:
- Parameters: Mean diameter , standard deviation . Tolerance limits: to .
- Proportion under : Using standard normal tables, .
- Probability exceeding :.
- Proportion meeting specifications ( to ): Thus, approximately of ball bearings meet specifications.
- Middle of Diameters: Middle leaves () in each tail. The critical -values corresponding to areas of and are . Middle range: .
- Percentile Calculation (Home Run Distance):
- To find the percentile rank of a home run (, ): Cumulative area to the left: . The home run is at the th percentile.
- Finding Distance at the th Percentile: For an area of to the left, the corresponding -score is approximately .
- Critical Value Notation ():
- The notation denotes the specific -score whose area under the standard normal curve to the right equals .
- For example, represents the -score with an area of to its right, which corresponds to the th percentile ( area to the left). Standard normal tables provide .
- Ball Bearing Factory Specifications:
Assessing Normality and Normal Approximations
Assessing Normality via Normal Probability Plots:
- Histograms drawn from small samples () do not reliably reflect the true shape of the underlying population distribution.
- A normal probability plot graphs observed sample values on the horizontal axis against their expected normal scores (-scores) on the vertical axis.
- Step-by-Step Procedure for Constructing a Normal Probability Plot:
- Step 1: Sort the observed data values in ascending order ().
- Step 2: For each sorted value at index (where ), compute the expected cumulative proportion : where represents the expected area under the standard normal curve to the left of the th observation.
- Step 3: Determine the expected normal score (-score) corresponding to each cumulative area using standard normal probability tables.
- Step 4: Plot the observed data values () on the horizontal axis and the corresponding expected -scores on the vertical axis.
- Evaluation of Linearity: If the plotted points form an approximately straight linear pattern, it is reasonable to conclude that the sample was drawn from a normally distributed population.
- Formal Decision Rule: Calculate the linear correlation coefficient between the observed data and expected -scores. If exceeds the critical value from Table VI for sample size , the assumption of population normality is supported.
- Worked Example (Statistics Exam Scores, ):
- Given scores: .
- With , denominator :
- ():
- ():
- ():
- ():
- ():
- ():
- ():
- Occupational Health Study (FEV/VC Ratio): An investigation evaluating seventeen () factory workers exposed to Bacillus subtilis enzyme measured the ratio of forced expiratory volume in one second (FEV) to vital capacity (VC) to determine if lung function ratios conform to a normal distribution via probability plotting.
The Normal Approximation to the Binomial Probability Distribution:
- When the number of trials is sufficiently large, the discrete binomial distribution can be approximated by a continuous normal distribution.
- Rule of Thumb Criterion:
- Approximation Parameters:
- Mean:
- Standard deviation:
- Flight Cancellations Evaluation (LaGuardia Airport):
- Data: Cancellation rate is () with a sample of flights.
- Checking the condition:
- Because , the condition is violated, and the normal approximation cannot be used.
- Valentine's Day Gifts for Pet Dogs:
- Data: () of American households buy gifts for pet dogs; sample size .
- Checking the condition: The condition is satisfied; the normal approximation is valid.
- Expected Number (Mean):
- Standard Deviation:
- Probability of 20 or More Gifts (): Using the standardizing transformation (with continuity correction or directly):
- Probability of More Than 25 Gifts ():