Discrete and Continuous Random Variables, Distributions, Expectation, and Variance

Foundations of Random Variables

  • Concept and Motivation:

    • In many probability experiments, the primary interest lies in a specific numerical quantity derived from an outcome rather than the full descriptive outcome itself.
    • Coin Tossing Example: When flipping multiple coins, interest usually centers on the total count of heads observed rather than the specific ordered sequence of heads and tails (e.g., HHHTHHHT vs. HTHHHTHH).
    • Dice Rolling Example: When tossing two dice, interest typically lies in the sum of the face values. For instance, knowing the total sum is 77 is often more relevant than distinguishing whether the individual dice showed (1,6)(1, 6), (2,5)(2, 5), (3,4)(3, 4), (4,3)(4, 3), (5,2)(5, 2), or (6,1)(6, 1).

  • Formal Definition:

    • A Random Variable is a functional mapping that assigns a single real number to each outcome/element in a sample space SS.
    • Because the numerical value taken by a random variable depends on the outcome of a probabilistic experiment, probabilities can be assigned directly to its possible values.

  • Example: Tossing Three Fair Coins:

    • Let YY denote the random variable representing the total number of heads obtained in three flips of a fair coin.
    • The sample space SS consists of 88 equally likely outcomes:         S={HHH,HHT,HTH,HTT,THH,THT,TTH,TTT}S = \{HHH, HHT, HTH, HTT, THH, THT, TTH, TTT\}
    • Mapping outcomes to values yy of YY:
      • HHH→y=3HHH \rightarrow y = 3
      • HHT,HTH,THH→y=2HHT, HTH, THH \rightarrow y = 2
      • HTT,THT,TTH→y=1HTT, THT, TTH \rightarrow y = 1
      • TTT→y=0TTT \rightarrow y = 0
    • Probability distribution P(Y=y)P(Y = y) for each possible value:
      • P(Y=0)=18P(Y = 0) = \frac{1}{8}
      • P(Y=1)=38P(Y = 1) = \frac{3}{8}
      • P(Y=2)=38P(Y = 2) = \frac{3}{8}
      • P(Y=3)=18P(Y = 3) = \frac{1}{8}

  • Classification of Random Variables:

    • Discrete Random Variables: Represent "count data", such as the number of defective items in a sample or the number of highway fatalities per year in a given region.
    • Continuous Random Variables: Represent "measured data", such as heights, weights, temperatures, distances, and life periods or operational durations.

Discrete Random Variables and Probability Distributions

  • Definition:

    • A random variable XX is defined as discrete if its set of possible values is finite or countably infinite.

  • Probability Mass Function (PMF):

    • The PMF (also called the discrete probability distribution function) f(x)f(x) defines the probability associated with each specific value xx of the discrete random variable XX.
    • A function f(x)f(x) is a valid PMF if and only if it satisfies the following three conditions:
      1. f(x)≥0f(x) \ge 0 for all possible values xx
      2. P(X=x)=f(x)P(X = x) = f(x)
      3. ∑xf(x)=1\sum_{x} f(x) = 1

  • Cumulative Distribution Function (CDF):

    • The CDF F(x)F(x) of a discrete random variable XX with PMF f(x)f(x) measures the cumulative probability that XX assumes a value less than or equal to xx:         F(x)=P(X≤x)=∑t≤xf(t),for −∞<x<∞F(x) = P(X \le x) = \sum_{t \le x} f(t), \quad \text{for } -\infty < x < \infty
    • The graph of a discrete CDF is a step function that increases monotonically from 00 to 11, with jump discontinuities occurring at each possible value xix_i of height equal to f(xi)f(x_i).

  • Worked Example 1: Distribution of Random Variable MM:

    • Given PMF for discrete variable MM:
      • f(0)=P(M=0)=13f(0) = P(M = 0) = \frac{1}{3}
      • f(1)=P(M=1)=12f(1) = P(M = 1) = \frac{1}{2}
      • f(3)=P(M=3)=16f(3) = P(M = 3) = \frac{1}{6}
    • Constructing the Cumulative Distribution Function F(m)F(m):         F(m)={0,for m<013,for 0≤m<156,for 1≤m<31,for m≥3F(m) = \begin{cases} 0, & \text{for } m < 0 \\ \frac{1}{3}, & \text{for } 0 \le m < 1 \\ \frac{5}{6}, & \text{for } 1 \le m < 3 \\ 1, & \text{for } m \ge 3 \end{cases}

  • Worked Example 2: Defective Computers Purchase:

    • A shipment of 88 computers contains 33 defective units and 55 non-defective units. A school randomly purchases 22 computers from the shipment without replacement.
    • Let XX be the random variable representing the number of defective computers purchased.
    • Possible values of XX: x∈{0,1,2}x \in \{0, 1, 2\}.
    • PMF derivation using combination formulas:
      • f(0)=P(X=0)=(30)(52)(82)=1×1028=1028f(0) = P(X = 0) = \frac{\binom{3}{0}\binom{5}{2}}{\binom{8}{2}} = \frac{1 \times 10}{28} = \frac{10}{28}
      • f(1)=P(X=1)=(31)(51)(82)=3×528=1528f(1) = P(X = 1) = \frac{\binom{3}{1}\binom{5}{1}}{\binom{8}{2}} = \frac{3 \times 5}{28} = \frac{15}{28}
      • f(2)=P(X=2)=(32)(50)(82)=3×128=328f(2) = P(X = 2) = \frac{\binom{3}{2}\binom{5}{0}}{\binom{8}{2}} = \frac{3 \times 1}{28} = \frac{3}{28}
    • CDF Formulation F(x)F(x):
      • For x<0x < 0: F(x)=0F(x) = 0
      • For 0≤x<10 \le x < 1: F(x)=f(0)=1028F(x) = f(0) = \frac{10}{28}
      • For 1≤x<21 \le x < 2: F(x)=f(0)+f(1)=1028+1528=2528F(x) = f(0) + f(1) = \frac{10}{28} + \frac{15}{28} = \frac{25}{28}
      • For x≥2x \ge 2: F(x)=f(0)+f(1)+f(2)=2528+328=1F(x) = f(0) + f(1) + f(2) = \frac{25}{28} + \frac{3}{28} = 1

  • Worked Example 3: Given Discrete Distribution and Graphs:

    • Given PMF for random variable XX:
      • f(0)=116f(0) = \frac{1}{16}
      • f(1)=14=416f(1) = \frac{1}{4} = \frac{4}{16}
      • f(2)=38=616f(2) = \frac{3}{8} = \frac{6}{16}
      • f(3)=14=416f(3) = \frac{1}{4} = \frac{4}{16}
      • f(4)=116f(4) = \frac{1}{16}
    • Corresponding CDF F(x)F(x) piecewise definition:         F(x)={0,for x<0116,for 0≤x<1516,for 1≤x<21116,for 2≤x<31516,for 3≤x<41,for x≥4F(x) = \begin{cases} 0, & \text{for } x < 0 \\ \frac{1}{16}, & \text{for } 0 \le x < 1 \\ \frac{5}{16}, & \text{for } 1 \le x < 2 \\ \frac{11}{16}, & \text{for } 2 \le x < 3 \\ \frac{15}{16}, & \text{for } 3 \le x < 4 \\ 1, & \text{for } x \ge 4 \end{cases}
    • Visual representation comparing the spike plot of the PMF and the step function plot of the CDF:

    Plot of PMF and Plot of CDF

  • Mean or Expected Value of a Discrete Random Variable:
    • The expected value E(X)E(X), denoted as μ\mu, represents the weighted average of the values taken by XX:         μ=E(X)=∑xxf(x)\mu = E(X) = \sum_{x} x f(x)
    • Sample Component Example: Let XX be the number of good components in a sample with PMF f(0)=120f(0) = \frac{1}{20}, f(1)=320f(1) = \frac{3}{20}, f(2)=1020f(2) = \frac{10}{20}, and f(3)=620f(3) = \frac{6}{20}. The expected value is:         E(X)=(0)(120)+(1)(320)+(2)(1020)+(3)(620)=0+320+2020+1820=4120=2.05E(X) = (0)\left(\frac{1}{20}\right) + (1)\left(\frac{3}{20}\right) + (2)\left(\frac{10}{20}\right) + (3)\left(\frac{6}{20}\right) = 0 + \frac{3}{20} + \frac{20}{20} + \frac{18}{20} = \frac{41}{20} = 2.05

Continuous Random Variables and Density Functions

  • Definition:

    • A random variable XX is continuous if it can take on any real value along a continuous scale or numerical interval.

  • Probability Density Function (PDF):

    • A function f(x)f(x) is a probability density function for a continuous random variable XX defined over the real numbers R\mathbb{R} if:
      1. f(x)≥0f(x) \ge 0 for all x∈Rx \in \mathbb{R}
      2. ∫−∞+∞f(x) dx=1\int_{-\infty}^{+\infty} f(x)\,dx = 1
      3. P(a<X<b)=∫abf(x) dxP(a < X < b) = \int_{a}^{b} f(x)\,dx

  • Common Density Function Shapes:

    • Density curves can take various functional forms depending on the underlying distribution (e.g., positively or negatively skewed curves, uniform linear functions, symmetric normal bell curves, or parabolic shapes).

    Typical Density Function Graphs

  • Probability at Single Point Values:

    • For any continuous random variable XX, the probability of assuming any exact single point value aa is identically zero:         P(X=a)=∫aaf(x) dx=0P(X = a) = \int_{a}^{a} f(x)\,dx = 0
    • Direct Implication: The probability distribution of a continuous random variable cannot be listed in tabular form.

  • Probabilities over Intervals:

    • Because individual points carry zero probability mass, inclusion or exclusion of interval endpoints does not affect interval probabilities:         P(a≤X≤b)=P(a≤X<b)=P(a<X≤b)=P(a<X<b)=∫abf(x) dxP(a \le X \le b) = P(a \le X < b) = P(a < X \le b) = P(a < X < b) = \int_{a}^{b} f(x)\,dx
    • Note: This property holds strictly for continuous random variables and is false for discrete random variables.
    • Geometric Interpretation: The probability P(a<X<b)P(a < X < b) equals the area under the density curve f(x)f(x) bounded between the ordinates at x=ax = a and x=bx = b, lying above the horizontal axis.

    Probabilities for Intervals Area under Curve

  • Cumulative Distribution Function (CDF):

    • The CDF F(x)F(x) of a continuous random variable XX with PDF f(x)f(x) is defined as:         F(x)=P(X≤x)=∫−∞xf(t) dt,for −∞<x<∞F(x) = P(X \le x) = \int_{-\infty}^{x} f(t)\,dt, \quad \text{for } -\infty < x < \infty
    • Computing interval probabilities using the CDF:         P(a<X<b)=∫abf(x) dx=F(b)−F(a)P(a < X < b) = \int_{a}^{b} f(x)\,dx = F(b) - F(a)

  • Mean or Expected Value of a Continuous Random Variable:

    • The expected value E(X)E(X), denoted as μ\mu, is given by the integral:         μ=E(X)=∫−∞+∞xf(x) dx\mu = E(X) = \int_{-\infty}^{+\infty} x f(x)\,dx

Variance and Standard Deviation of Random Variables

  • Definition of Variance:

    • The variance of a random variable XX, denoted as Var(X)\text{Var}(X) or σ2\sigma^2, measures the expected squared deviation of XX from its mean μ\mu:         σ2=Var(X)=E((X−μ)2)\sigma^2 = \text{Var}(X) = E\left((X - \mu)^2\right)

  • Formulas by Random Variable Type:

    • Discrete Random Variable:         σ2=Var(X)=∑x(x−μ)2f(x)\sigma^2 = \text{Var}(X) = \sum_{x} (x - \mu)^2 f(x)
    • Continuous Random Variable:         σ2=Var(X)=∫−∞+∞(x−μ)2f(x) dx\sigma^2 = \text{Var}(X) = \int_{-\infty}^{+\infty} (x - \mu)^2 f(x)\,dx

  • Alternative Computational Formula:

    • Expanding the squared term inside the expectation yields the shortcut formula:         σ2=Var(X)=E(X2)−μ2\sigma^2 = \text{Var}(X) = E(X^2) - \mu^2
    • Where E(X2)E(X^2) is computed as:
      • Discrete: E(X2)=∑xx2f(x)E(X^2) = \sum_{x} x^2 f(x)
      • Continuous: E(X2)=∫−∞+∞x2f(x) dxE(X^2) = \int_{-\infty}^{+\infty} x^2 f(x)\,dx

  • Definition of Standard Deviation:

    • The standard deviation σ\sigma of a random variable XX is defined as the positive square root of its variance:         σ=Var(X)=σ2\sigma = \sqrt{\text{Var}(X)} = \sqrt{\sigma^2}

  • Detailed Worked Example: Official Business Automobiles Usage:

    • Problem: Let XX represent the number of automobiles used for official business on any given workday. The PMF of XX is given as follows:         | xx | f(x)f(x) |         | :--- | :--- |         | 11 | 0.30.3 |         | 22 | 0.40.4 |         | 33 | 0.30.3 |         Find the mean, variance, and standard deviation of XX.
    • Step 1: Compute Mean μ\mu:         μ=E(X)=∑xxf(x)=(1)(0.3)+(2)(0.4)+(3)(0.3)=0.3+0.8+0.9=2.0\mu = E(X) = \sum_{x} x f(x) = (1)(0.3) + (2)(0.4) + (3)(0.3) = 0.3 + 0.8 + 0.9 = 2.0
    • Step 2: Compute Variance using Definition Formula σ2=∑(x−μ)2f(x)\sigma^2 = \sum (x - \mu)^2 f(x):
      • For x=1x = 1: (1−2)2(0.3)=(−1)2(0.3)=0.3(1 - 2)^2 (0.3) = (-1)^2 (0.3) = 0.3
      • For x=2x = 2: (2−2)2(0.4)=(0)2(0.4)=0(2 - 2)^2 (0.4) = (0)^2 (0.4) = 0
      • For x=3x = 3: (3−2)2(0.3)=(1)2(0.3)=0.3(3 - 2)^2 (0.3) = (1)^2 (0.3) = 0.3
      • Summing deviations: σ2=0.3+0+0.3=0.6\sigma^2 = 0.3 + 0 + 0.3 = 0.6
    • Step 3: Compute Variance using Computational Formula σ2=E(X2)−μ2\sigma^2 = E(X^2) - \mu^2:
      • Compute E(X2)E(X^2):             E(X2)=∑xx2f(x)=(12)(0.3)+(22)(0.4)+(32)(0.3)=(1)(0.3)+(4)(0.4)+(9)(0.3)=0.3+1.6+2.7=4.6E(X^2) = \sum_{x} x^2 f(x) = (1^2)(0.3) + (2^2)(0.4) + (3^2)(0.3) = (1)(0.3) + (4)(0.4) + (9)(0.3) = 0.3 + 1.6 + 2.7 = 4.6
      • Subtract μ2\mu^2:             σ2=4.6−(2.0)2=4.6−4.0=0.6\sigma^2 = 4.6 - (2.0)^2 = 4.6 - 4.0 = 0.6
    • Step 4: Compute Standard Deviation σ\sigma:         σ=0.6≈0.775\sigma = \sqrt{0.6} \approx 0.775