Biostatistics Conditional Probability, Random Variables, and Normal Distribution Lecture 3 (9/17)

Conditional Probability and Joint/M marginal/Probability Contexts

• We are analyzing probabilities in a table setup with three drug groups (desipramine, lithium, placebo) and relapse outcomes.
• Key terms:
  • Marginal probability: the probability of an event ignoring any conditioning, calculated from the whole table.
  • Joint probability: the probability of two events happening together (intersection).
  • Conditional probability: the probability of an event A given that event B has happened, written as $P(A\mid B) = \frac{P(A \cap B)}{P(B)}.$
• Example setup (cocaine relapse study): 72 chronic cocaine users randomly assigned to three groups (24 per group).
  • Counts relapsed per drug group: desipramine: 10 relapsed; lithium: 18 relapsed; placebo: 20 relapsed.
  • Totals: 24 relapsed in each drug group; overall relapsed total = 48; non-relapsed total = 24.

Marginal probability of relapse

• Overall probability someone relapsed: $P(\text{Relapse}) = \frac{48}{72} = 0.667.$

Joint probability

• Probability a patient received desipramine and relapsed: $P(\text{Relapse} \cap \text{Desipramine}) = \frac{10}{72} \approx 0.139.$

Conditional probabilities given treatment

• Probability of relapse given desipramine:
  $P(\text{Relapse} \mid \text{Desipramine}) = \frac{P(\text{Relapse} \cap \text{Desipramine})}{P(\text{Desipramine})} = \frac{10/72}{24/72} = \frac{10}{24} \approx 0.417.$
• Probability of relapse given lithium:
  $P(\text{Relapse} \mid \text{Lithium}) = \frac{18}{24} = 0.75.$
• Probability of relapse given placebo:
  $P(\text{Relapse} \mid \text{Placebo}) = \frac{20}{24} \approx 0.833.$

Conditional probabilities in the reverse direction

• Probability of having desipramine given relapse:
  $P(\text{Desipramine} \mid \text{Relapse}) = \frac{P(\text{Relapse} \cap \text{Desipramine})}{P(\text{Relapse})} = \frac{10/72}{48/72} = \frac{10}{48} \approx 0.2083.$
• Probability of having lithium given relapse:
  $P(\text{Lithium} \mid \text{Relapse}) = \frac{18}{48} = 0.375.$
• Probability of having placebo given relapse:
  $P(\text{Placebo} \mid \text{Relapse}) = \frac{20}{48} \approx 0.417.$

Notes on independence and dependence

• If A and B are independent, then: $P(A \cap B) = P(A)P(B).$
• If A and B are not independent, then: $P(A \cap B) = P(A \mid B)P(B).$
• Order matters conceptually: if B happens first, use $P(A|B)P(B)$; if A happens first, use $P(B|A)P(A).$
• Practical takeaway: independence means knowing B tells you nothing about A; e.g., whether gender affects being a social science major in a given class (in the example, probability remains 0.60 for both genders).

Quick example illustrating independence (gender and major)

• Suppose 100 students: 50 female, 50 male; 60 social science majors, 40 not.
• Female: 30 social science, 20 not; Male: 30 social science, 20 not.
• Marginal probability of social science: $P(SS) = \frac{60}{100} = 0.60.$
• Conditional on female: $P(SS|Female) = \frac{30}{50} = 0.60.$
• Conditional on male: $P(SS|Male) = \frac{30}{50} = 0.60.$
• Since the condition does not change the probability, A and B (gender and major) are independent in this example.

Random Variables, Discrete vs Continuous, and Expectation

• A random variable X assigns a numeric value to each outcome of a random experiment.
  • X is the variable; values are x; the probability that X equals a specific value x is p(x).
• Two types of random variables:
  • Discrete: X takes countable values (often integers), e.g., number of children, number of credits. Probability mass function (PMF) p(x).
  • Continuous: X takes an interval of values (often real numbers), e.g., height, weight, time. Probability density function (PDF) f(x); probabilities are areas under the curve.
• The mean (expected value) of X:
  • Discrete: $\mu = \mathbb{E}[X] = \sum_{x} x \; p(x).$
  • Continuous: $\mathbb{E}[X] = \int_{-\infty}^{\infty} x f(x) \, dx.$
• Worked example (discrete): a casino card game
  • Outcomes and winnings:
  • Draw a heart (non-ace): win $1; probability $\frac{12}{52}$.
  • Draw an ace (including ace of hearts): win $5; probability $\frac{4}{52}$.
  • Draw the king of spades: win $10; probability $\frac{1}{52}$.
  • Any other card: win $0; probability $\frac{35}{52}$.
  • Expected value: $\mathbb{E}[X] = 1\cdot\frac{12}{52} + 5\cdot\frac{4}{52} + 10\cdot\frac{1}{52} + 0\cdot\frac{35}{52} = \frac{12+20+10}{52} = \frac{42}{52} \approx 0.8077.$
  • Interpretation: on average, you win about $0.81 per play; since you pay $1 to play, the net expectation is about -$0.19 per game.
• PMF vs PDF intuition:
  • Discrete distributions have probabilities at specific points; continuous distributions assign probabilities to intervals via areas.
  • For a continuous X, P(X = x) = 0 for any exact x (the area of a razor-thin point is zero).
• Visual intuition: distributions are often shown as bars (discrete) or a smooth curve (continuous PDFs).

The Normal Distribution and Standardization

• The normal distribution is bell-shaped (Gaussian), unimodal, symmetric, and is denoted by $N(\mu, \sigma)$ where:
  • $\mu$ is the mean (location of the center).
  • $\sigma$ is the standard deviation (spread).
• Standard normal distribution: $Z \sim N(0,1)$. Any normal can be standardized by the z-score:
  • $Z = \frac{X - \mu}{\sigma}.$
• Why standardize? to compare different normally distributed quantities on a common scale.
• Example: test scores
  • SAT: approximately normal with mean $\mu<em>{SAT} = 1500$ and standard deviation $\sigma</em>{SAT} = 300$.
  • ACT: approximately normal with mean $\mu<em>{ACT} = 21$ and standard deviation $\sigma</em>{ACT} = 5$.
• Z-score examples (comparative interpretation):
  • Pam: SAT 1800 -> $Z_{Pam} = \frac{1800-1500}{300} = 1.0.$
  • Jim: ACT 24 -> $Z_{Jim} = \frac{24-21}{5} = 0.6.$
  • Higher z-score means better performance relative to peers for that test, but cross-test comparison requires standardization (hence z-scores).
• Percentiles and the cumulative distribution function (CDF):
  • Percentile for a value x is the probability that a random observation is below x: $\text{Percentile}(x) = P(X \le x).$
  • For the standard normal, use the CDF $\Phi(z) = P(Z \le z).$
• How to compute percentiles with R (conceptual): P(X ≤ x) for a normal with mean $\mu$ and sd $\sigma$ can be computed as $P(X \le x) = \Phi\left(\frac{x-\mu}{\sigma}\right).$
  • Example: Pam's SAT score 1800 corresponds to a z-score of 1.00, so percentile ≈ $\Phi(1.0) \approx 0.8413$ (84.13%).
  • Jim's ACT score 24 corresponds to z = 0.60, so percentile ≈ $\Phi(0.60) \approx 0.7257$ (≈72.6%).
• Calculating probabilities with the normal distribution (intervals):
  • Probability X between a and b: $P(a \le X \le b) = \Phi\left(\frac{b-\mu}{\sigma}\right) - \Phi\left(\frac{a-\mu}{\sigma}\right).$
  • If you want probabilities to the right of a value: $P(X \ge x) = 1 - \Phi\left(\frac{x-\mu}{\sigma}\right).$
• How to use R's PNORM (conceptual):
  • Notation: $\text{pnorm}(q, \text{mean}, \text{sd}, \text{lower.tail} = TRUE)$ returns $P(X \le q)$ for a normal with the given mean and sd (left tail by default).
  • Examples in practice:
  • Pam: pnorm(1800, mean = 1500, sd = 300) ≈ 0.8413.
  • Jim: pnorm(0.6, mean = 0, sd = 1) ≈ 0.7257.
  • Probability to the right of Pam: 1 - pnorm(1.0, mean = 0, sd = 1) ≈ 0.1587.
• Two key practical tricks:
  • When using z-scores, results are often rounded to two decimal places (e.g., z = 1.00, z = 0.60).
  • If you need the probability to the right, use $1 - P(X \le x)$ or equivalently $1 - \text{pnorm}(z, 0, 1).$

Working with a Continuous Example: Ketchup Bottle QA

• Setup: The amount of ketchup in a bottle is normally distributed with mean $\mu = 36$ ounces and standard deviation $\sigma = 0.11$ ounces.
• Quality control rule: Fail if X < 35.8 or X > 36.2. Pass if X is in [35.8, 36.2].
• Convert to z-score for a quick check:
  • $Z = \frac{X - \mu}{\sigma} = \frac{35.8 - 36}{0.11} = -1.82.$
  • Probability X < 35.8: $P(X < 35.8) = P(Z < -1.82) \approx 0.0344.$ (Using PNORM or standard normal table; note small rounding differences with exact values.)
• Exact vs approximate probabilities:
  • Using the exact mean and sd in PNORM yields $P(X < 35.8) \approx 0.0345.$ (slightly different in the fourth decimal place due to rounding and precision differences)
• Probability of passing quality: P(35.8 ≤ X ≤ 36.2) = Φ(1.82) - Φ(-1.82) ≈ 0.9656 - 0.0344 = 0.9312 (93.12%).
• Practical interpretation:
  • If you want the probability of being below 35.8, use the left tail; if you want the probability of staying within the acceptable band, compute the area between the two z-values.
  • The exact probability vs rounded probability may differ slightly in the 4th decimal place, but the overall conclusion is the same.

Important Practical Rules and Tips

• For continuous distributions:
  • P(X = x) = 0 for any exact value x. Probabilities are always computed over intervals, via areas under the curve.
  • The probability of being within an interval corresponds to the area under the PDF between the endpoints.
• Normal distribution basics:
  • Standard normal: $Z \sim N(0,1)$ with pdf $\phi(z) = \frac{1}{\sqrt{2\pi}} e^{-z^2/2}$ and CDF $\Phi(z) = \int_{-\infty}^{z} \phi(t) dt$.
  • Any normal X ~ N(\mu, σ) can be transformed to Z by $Z = \frac{X - \mu}{σ}$ and probabilities computed via Φ.
• Z-scores and percentiles:
  • Z-score tells you how many standard deviations an observation is from the mean.
  • Percentiles are computed as the left-tail probability up to that z-score (area to the left under the standard normal curve).
  • When reporting z-scores, round to two decimal places (e.g., 0.60, 1.00).
• Using software (R) for normal probabilities:
  • PNORM(q, mean, sd) computes $P(X\le q)$ for X ~ N(mean, sd).
  • To get the right-tail probability, use $1 - \text{pnorm}(q, mean, sd)$.
• Conceptual takeaways:
  • Marginal vs conditional probabilities:Marginals summarize across all groups; conditionals incorporate information about a specific subset.
  • Independence vs dependence: Independence implies P(A|B) = P(A); otherwise, P(A∩B) = P(A|B)P(B) or P(B|A)P(A).
  • The normal distribution is a central model in statistics because many real-world quantities are approximately normal due to the central limit theorem; it provides a convenient way to compare different measurements on a standard scale via z-scores.

Quick Recap of Notation and Key Formulas

• Conditional probability:
  $P(A\mid B) = \frac{P(A\cap B)}{P(B)}.$
• Joint probability under independence:
  $P(A\cap B) = P(A)P(B)$ if A and B are independent.
• Joint probability without independence:
  $P(A\cap B) = P(A\mid B)P(B) = P(B\mid A)P(A).$
• Marginal probability: e.g., overall relapse rate
  $P(\text{Relapse}) = \frac{48}{72}.$
• Expected value (discrete):
  $\mu = \mathbb{E}[X] = \sum_x x \; p(x).$
• Expected value (continuous):
  $\mathbb{E}[X] = \int_{-\infty}^{\infty} x f(x) \; dx.$
• Normal distribution notation:
  $X \sim N(\mu, \sigma^2)$ or $N(\mu, \sigma)$ for shorthand; standard normal $Z \sim N(0,1)$ with $Z = \frac{X-\mu}{\sigma}.$
• Z-score:
  $z = \frac{X - \mu}{\sigma}.$
• Probability between a and b:
  $P(a \le X \le b) = \Phi\left(\frac{b-\mu}{\sigma}\right) - \Phi\left(\frac{a-\mu}{\sigma}\right).$
• PNORM function (concept):
  $\text{pnorm}(q, \text{mean}, \text{sd}) = P(X \le q).$
• Probability of being exactly a value in a continuous distribution is 0:
  $P(X = x) = 0.$