Probability and Statistics Notes

Experiment, Outcomes, and Sample Space

• Experiment: An act or process that results in one and only one of many observations.
• Outcome: The result of the experiment (observation).
• Sample Space: The collection of all possible outcomes for an experiment.
  • Denoted by S.
  • Elements of a sample space are called sample points.
• Example:
  • Experiment: Inspecting a tennis ball.
  • Outcomes: "good" or "defective".
  • Sample space: S = {good, defective}.

Examples of Experiments, Outcomes, and Sample Spaces

• Toss a coin once:
  • Outcomes: Head, Tail
  • Sample Space: S = {Head, Tail}
• Roll a die once:
  • Outcomes: 1, 2, 3, 4, 5, 6
  • Sample Space: S = {1, 2, 3, 4, 5, 6}
• Toss a coin twice:
  • Outcomes: HH, HT, TH, TT
  • Sample Space: S = {HH, HT, TH, TT}
• Birth of a baby:
  • Outcomes: Boy, Girl
  • Sample Space: S = {Boy, Girl}
• Take a test:
  • Outcomes: Pass, Fail
  • Sample Space: S = {Pass, Fail}
• Select a student:
  • Outcomes: Male, Female
  • Sample Space: S = {Male, Female}

Describing a Sample Space

• Venn Diagram: A picture (closed geometric shape) that depicts all possible outcomes for an experiment.
• Tree Diagram: Each outcome is represented by a branch of the tree.
• Example: One toss of a coin.
  • Venn Diagram: A circle or rectangle with H and T inside.
  • Tree Diagram: A tree with two branches, one for H and one for T.
• Example: Two tosses of a coin.
  • Venn Diagram: A circle or rectangle with HH, HT, TH, and TT inside.
  • Tree Diagram: A tree where the first toss has two branches (H, T), and each of those branches has two more branches (H, T) for the second toss, resulting in HH, HT, TH, TT.
  • This experiment can be split into two parts: the first toss and the second toss.

Venn Diagram and Relationships among Sets

• All elements of A
• Complement Ac: All elements that are NOT in A
• Union (or) $A \cup B$: Sample space for A and B
• Intersection (and) $A \cap B$

Simple and Compound Events

• Event: Consists of one or more of the outcomes of an experiment. It's a collection of one or more outcomes.
• Simple (or Elementary) Event: Each of the final outcomes for an experiment. It includes one and only one outcome and denoted by Ei.
• Compound Event (or Composite): Consists of more than one outcome. It is a collection of more than one outcome for an experiment, denoted by A, B, C, …, or A1, A2, A3, …., B1, B2, B3, ….., C1, C2, C3 and so on.

Simple and Compound Events Example

• A sample space, denoted S, of an experiment includes all possible outcomes of the experiment.
• For example, a sample space containing letter grades is: $S = {A, B, C, D, F}$
• An event is a subset of the sample space.
• The event “passing grades” is a subset of S (compound event). {A, B, C, D}
• The simple event “failing grades” is a subset of S. {F}

The Properties of Probabilities

• Probability: A numerical measure of the likelihood that a specific event will occur.
• Property 1: The probability of an event always lies in the range 0 to 1.
  • $0 \le P(E) \le 1$ for any simple event E
  • $0 \le P(A) \le 1$ for any compound event A
  • 0 = Impossible, 1 = Certain, 0.5 = 50/50 chance of occuring.

The Properties of Probabilities

• Property 2: The sum of the probabilities of all simple events (or final outcomes), for an experiment, denoted by $\sum P(Ei)$, is always 1.
  • $\sum P(Ei) = P(E1) + P(E2) + P(E3) + … = 1$
  • For the experiment of one toss of a coin: $P(H) + P(T) = 1$
  • For the experiment of two tosses of a coin: $P(HH) + P(HT) + P(TH) + P(TT) = 1$

Three Conceptual Approaches to Probabilities

• Classical Probability: logical analysis.
• Empirical Probability: frequency concept of occurrence.
• Subjective Probability: personal and subjective judgment.
• Objective probabilities.
• Subjective probability.

Classical Probabilities

• The classical probability rule is applied to compute the probabilities of events for an experiment all of whose outcomes are equally likely.
• Two or more outcomes (or events) that have the same probability of occurrence are said to be equally likely outcomes (or events).
• Classical Probability Rule: $P(A) = \frac{\text{Number of outcomes favorable to A}}{\text{Total number of outcomes for the experiment}}$
• $P(Ei) = \frac{1}{\text{Total number of outcomes for the experiment}}$

Classical Probabilities Examples

• Example 1: Find the probability of obtaining a head and the probability of obtaining a tail for one toss of a coin.
  • $P(H) = \frac{1}{2} = 0.5$
  • $P(T) = \frac{1}{2} = 0.5$
• Example 2: Find the probability of obtaining an even number in one roll of a die.
  • $A = {2, 4, 6}$
  • $P(A) = \frac{3}{6} = 0.5$
• Example 3: A club has 100 members, of whom 60 are men and 40 are women. Suppose one of these members is randomly selected to be the president of the club. What is the probability that a woman is selected?
  • If A – a woman is selected, then $P(A) = \frac{40}{100} = 0.4$
  • Because the selection is to be made randomly, each of the 100 members of the club has the same probability of being selected.

Relative Frequency Concept of Probability

• The probabilities cannot be computed using the classical probability rule because the various outcomes for the corresponding experiments are not equally likely.
• Each of this experiments can be performed again and again to generate data.
• In such cases, to calculate probabilities, we either use past data or generate new data by performing the experiment a large number of times.
• If the experiment is repeated n times and the event A is observed f times, then, according to the RELATIVE FREQUENCY CONCEPT OF PROBABILITY: $P(A) = \frac{f}{n}$
• Examples:
  • The probability that the next car that comes out of an auto factory is a “lemon.”
  • The probability that a randomly selected family owns two cars.
  • The probability that an 80-year-old person will live at least one more year.

Relative Frequency Concept of Probability EXAMPLE

• P(next car is a lemon) = 0.02
• P(next car is good) = 0.98
• Relative frequencies are not probabilities but approximate probabilities.
• However, if the experiment is repeated again and again, this approximate probability of an outcome obtained from the relative frequency will approach the actual or mathematical probability of that outcome. This is called Law of Large Numbers.

PROBLEM

Subjective Probability

• Sometimes we face experiments that neither have equally likely outcomes nor can be repeated to generate data.
• The probability assigned to the event in such cases is called subjective probability.
• These probabilities are based on subjective judgment, experience, information, or preferences.
• For example, consider the following probabilities of events:
  1. The probability that Carol, who is taking statistics, will earn A in this course.
  2. The probability that the Dow Jones Industrial average will be higher at the end of next trading day.
  3. The probability that the “XXX” team will win the Super Bowl next season.

Counting Rule

• If an experiment consists of three steps and if the first step can result in m outcomes, the second step in n outcomes, and the third step is k outcomes, then:
• Total outcomes for the experiment = m . n . k (Counting Rule)
• EXAMPLES:
  1. If we toss a coin three times, how many possible outcomes are possible?
     Total outcomes for the experiment = 2 . 2 . 2 = 8
  2. A car buyer can choose between a fixed or a variable interest rate and can also choose a payment period of 36 months, 48 months, or 60 months. How many possible outcomes are possible?
     Total outcomes for the experiment = 2 . 3 = 6
  3. A National Football League team will play 16 games during a regular season. Each game can result in one of three outcomes: a win, a loss, or a tie. How many possible outcomes are possible?
     Total outcomes for the experiment = 3 . 3 …… 3 = $3^{16}$ = 43,046,721

Counting and Arrangement – Permutations

• An arrangement of a set of n distinct objects in a given order r (r objects are chosen) is called a permutation.
• Example: How many 3-digit numbers can be formed from the digits: 2, 3, 4, 5, 6, 7? (* we do not repeat the digits)
  • 6.5.4 = 120
    $P(n, r) = \frac{n!}{(n - r)!}$
    
    P(6, 3) = \frac{6!}{(6 - 3)!} = \frac{6!}{3!} = 6 \cdot 5 \cdot 4 = 120

Counting and Possibilities – Combinations

• Combination Rule
  • Requirement:
    1. There are n distinct objects.
    2. We select r of n object (without replacement)
    3. The order does not count.(example: abc is the same as cba)

$C(n, r) = \frac{P(n, r)}{r!} = \frac{n!}{r!(n-r)!}$

Combinations. Examples

• Example 1: A small law firm has 16 employees and three are to be selected randomly to represent the company at the annual meeting of the American Association. How many different combinations of lawyers could be sent to the meeting?
  • n = 16 and r = 3
  • $<em>{16}C</em>3 = (16!) / (3!13!) = 560$
• Example 2: You plan to invest equal amounts of money in each of five business ventures. If you have 20 ventures from which to make a selection, how many different combinations of five ventures can be selected from the 20?
  • n = 20 and r = 5
  • $<em>{20}C</em>5 = (20!) / (5!15!) = 15,504$
• Example 3: In Florida’s state lottery game, called Pick-6 Lotto, you select six numbers of your choice from a set of numbers ranging from 1 to 53. What is the probability of winning lotto?
  • n = 53 and r = 6
  • $<em>{53}C</em>6 = (53!) / (6!47!) = 22,957,480$ – possible combinations
  • Since the Lotto balls are selected at random, each of these 22,957,480 combinations is equally likely to occur. Therefore, the probability of winning Lotto is = 1/(22,957,480) = 0.00000004356.
• Example 4: In how many ways can a committee considering of 3 men and 2 women be chosen from 7 men and 5 women?

Combinations Versus Permutations

• Example: The combination of the letters a, b, c, d taken 3 at a time are abc, abd, acd, bcd

• C(4,3).3! = P(4,3) or C(4,3) = P(4,3)/3!
• P(4,3) = 24 & C(4,3) = 4

Contingency Tables and Probabilities

• Example: 100 Employees of a company were asked whether they were in favor of or against a new paying policy:
• The contingency table gives the distribution of 100 employees based on two variables (characteristics): gender and opinion.

Marginal and Conditional Probabilities

• Marginal Probability is the probability of a single event without consideration of any other events. Marginal probability is also called simple probability.
  1. If one employee is selected at random from these 100 employees, then
  • P(male) = 60/100 = 0.6
  • P(female) = 40/100 = 0.4
  • P(in favor) = 19/100 = 0.19
  • P(against) = 81/100 = 0.81
  1. If one employee is selected at random from these 100 employees and we know that this employee is a male (or the event that the employee is a male has already occurred). What is the probability that the employee selected is in favor of new paying policy?
• P(in favor | male) = Number of man who are in favor/ Total number of male = 15/60 = 0.25

Conditional Probabilities

• Conditional Probability is the probability that the event will occur given that another event has already occurred or P(A | B) – the probability of A given that B has already occurred.
  3. Compute the conditional probability that a randomly selected employee is a female given that this employee is in favor of new paying policy.
• P(female | in favor) = Number of females who are in favor/ Total number of employees who are in favor = 4/19 = 0.21

Mutually Exclusive Events

• Events that cannot occur together are said to be mutually exclusive events.
• Example: Consider the following events for one roll of a die:
  • A = an even number is observed = {2, 4, 6}
  • B = an odd number is observed = {1, 3, 5}
• The events A and B have no common element. For one roll of a die, only one of the two events, A and B, can happen. Hence, these are two mutually exclusive events.

Independent Versus Dependent Events

• Two events are said to be independent events if the occurrence of one does not affect the probability of the occurrence of the other event. In other words, A and B are independent events if: either P(A | B) = P(A) or P(B | A) = P(B)
• Using the probability notation, the two events will be dependent if: either P(A | B) ≠ P(A) or P(B | A) ≠ P(B)

• P(D) = 15/100 = 0.15
• P(D | A) = 9/60 = 0.15

Complementary Events

• Two mutually exclusive events that taken together include all the outcomes for an experiment are called complementary events.
• Two complementary events are always mutually exclusive.
• P(A) + P(Ac) = 1
• P(A) = 1 – P(Ac)
• P(Ac) = 1 – P(A)
• A – the machine selected is defective
• Ac – the machine selected is not defective
• Example: There are a total of 120 professors at a college and 90 of them possess a PhD degree. If one professor is selected at random from this college, what are the two complementary events and their probabilities?

PROBLEM

• Example: A group of 1000 randomly selected adults were asked if they are in favor of or against abortion. The following table gives the results of this survey.
  1. If one person is selected at random these 1000 adults, find the probability that this person is:
  • a) in favor of abortion
  • b) against abortion
  • c) in favor of abortion given the person is a female
  • d) a male given the person is against abortion
  1. Are the events “male” and “in favor” mutually exclusive? What about the events “in favor” and “against”? Why and why not?
  2. Are the events “female” and “in favor” independent? Why and why not?
  3. If one person selected at random these 1000 adults, what are the two complementary events and their probabilities?

SOLUTIONS

1. P(I) = 55%

• P(A) = 45%
• P(I | F) = 320/570 = 56%
• P(M | A) = 200/450 = 44%

1. Yes: 230 = “males” and “in favor of”

• No

1. No: P(I) ≠ P(I | F)
2. P(I) + P(A) = 55% + 45% = 100%

• P(M) + P(F) = 43% + 57% = 100%

Union of Events and the Addition Rule

• $P(A \cup B) = P(A) + P(B) – P(A \cap B)$
• Union of Events is the collection of all outcomes that belong either to A or to B or to both A and B.
• Events A and B both occur: $A \cap B$
• A occurs or B occurs or both occur: $A \cup B$
• Addition Rule:

The Addition Rule for Two Mutually Exclusive Events

• A occurs or B occurs: $A \cup B$
• Events A and B both cannot occur. $A \cap B = 0$
• $P(A \cup B) = P(A) + P(B)$
• Addition Rule for mutually exclusive events:

Additional Rule - Example

• Example: A university president has proposed that all students must take a course in ethics as a requirement for graduation. Three hundred faculty members and students from this university were asked about their opinion on this issue.
  1. What is the probability that a randomly selected person from those 300 faculty members and students is a faculty member or is in favor of this proposal?
  2. What is the probability that a randomly selected person from those 300 faculty members and students is in favor of proposal or is neutral?

Additional Rule - Solution

1. A = the person selected is a faculty member

• B = the person selected is in favor of the proposal
• P(A) = 70/300 = 0.23
• P(B) = 135/300 = 0.45
• P(A and B) = 45/70 = 0.15
• Using the addition rule:
• P(A ∪ B) = P(A) + P(B) – P(A ∩ B) = 0.23 + 0.45 – 0.15 = 0.53

1. A = the person selected is in favor of the proposal

• B = the person selected is neutral
• P(A) = 135/300 = 0.45
• P(B) = 40/300 = 0.13
• The events A and B are mutually exclusive:
• P(A ∪ B) = P(A) + P(B) = 0.45 +0.13 = 0.58

Intersection of Events and Multiplication Rule

• The intersection of A and B represents the collection of all outcomes that are common to both A and B.
• The probability of the intersection of two events is called their joint probability.
• Conditional Probability:
  $P(A | B) = \frac{P(A \cap B)}{P(B)}$
  $P(A \cap B) = P(A) P(B | A)$
• Multiplication Rule:
  $P(A \cap B) = P(A) P(B | A) and P(A \cap B) = P(B) P(A | B)$
• If B has already occurred, the relevant portion of the sample space reduces to B.

Joint Probability: Example 1

• Example 1: The following table gives the classification of all employees of a company by gender and college degree. If one of these employees is selected at random for membership on the employee-management committee:
  1. What is the probability that this employee is a college graduate and a female?
  2. What is the probability that this employee is a college graduate or a female?

Example 1: Solution

1. P(G ∩ F) = 4/40 = 0.1 - joint probability

• P(G ∩ F) = P(G) P(F | G) = (11/40) (4/11) = 0.1 – joint probability

1. P(G ∪ F) = P(G) + P(F) – P(G ∩ F) =

• 11/40 + 13/40 – 4/40 = 0.5

Example 2

• Example 2: The probability that a randomly selected student from a college is a senior is 20%, and the joint probability that the student is a computer science major and a senior is 3%. Find the conditional probability that a student selected at random is a computer science major given that he/she is a senior?
• A = the student selected is a senior
• B = the student selected is a computer science major
• P(A) = 0.20, P(A ∩ B) = 0.03, P(B | A) = ?

Example 2: Solution

• Example 2: The probability that a randomly selected student from a college is a senior is 20%, and the joint probability that the student is a computer science major and a senior is 3%. Find the conditional probability that a student selected at random is a computer science major given that he/she is a senior?
• A = the student selected is a senior
• B = the student selected is a computer science major
• P(A) = 0.20, P(A ∩ B) = 0.03, P(B | A) = 0.03/0.20 = 0.15

Example 3

• Example 3: In a certain college, 25% of the students failed mathematics, 15% of the students failed chemistry, and 10% of the students failed both mathematics and chemistry. A student is selected at random:
  • a) if he/she failed chemistry, what is the probability that failed mathematics?
  • b) if he/she failed mathematics, what is the probability that failed chemistry?
  • c) What is the probability that he/she failed mathematics or chemistry?

Example 3: Solution

• P(M) = 0.25
• P(C) = 0.15
• P(М ∩ С) = 0.10
  • a) 0.10/0.15 = 2/3.
  • b) 0.10/0.25 = 2/5.
  • c) 0.25 + 0.15 – 0.10 = 0.30

Multiplication Rule for Independent Events

• The foregoing discussion of the Multiplication Rule is based on the assumption that the two events are dependant. Now suppose that the events A and B are independent. Two events are independent if and only if:
  • $P(A | B) = P(A)$
  • $P(B | A) = P(B)$
• General Multiplication Rule: $P(A \cap B) = P(A) \ast P(B | A)$
• Therefore:
• Multiplication Rule for Independent Events: $P(A \cap B) = P(A) P(B)$

Multiplication Rule for Independent Events - Example

• Example: The probability that a patient is allergic to penicillin is 20%. Suppose this drug is administered to three patients.
  1. Find the probability that all three of them are allergic to it.
  2. Find the probability that at least one of them is not allergic to it.
• A = the first patient is allergic
• B = the second patient is allergic
• C = the third patient is allergic

Solution

• A = the first patient is allergic
• B = the second patient is allergic
• C = the third patient is allergic
• These events are independent because whether or not one patient is allergic does not depend on whether or not any of the other patients is allergic or:
  $P(A \cap B \cap C) = P(A) P(B) P(C) = 0.2 \ast 0.2 \ast 0.2 = 0.008$
• G = all three patients are allergic
• H = at least one is not allergic
• Events G and H are two complementary events. Event G consists of the intersection of events A, B, and C or P(G) = $P(A \cap B \cap C)$ = 0.008
• Therefore P(H) = 1 – P(G) = 1 – 0.008 = 0.992

Joint Probability of Mutually Exclusive Events

• The joint probability of two mutually exclusive events is always 0.
• Example: Consider the following two events for an application filed by person to obtain a car loan:
  • A = event that the loan application is approved
  • B = event that the loan application is rejected.
• $P(A \cap B) = 0$

Rules of Probability

• The Addition Rule:
  $P(A \cup B) = P(A) + P(B) - P(A \cap B)$
• The Addition Rule for Two Mutually Exclusive Events:
  $P(A \cup B) = P(A) + P(B)$
• The Multiplication Rule:
  $P(A \cap B) = P(A) \ast P(B | A)$
• The Multiplication Rule for Independent Events:
  $P(A \cap B) = P(A) P(B)$
• Multiplication Rule for two mutually exclusive events:
  $P(A \cap B) = 0$