Random Variables

Chapter Three: Random Variables

1. Real Random Variable (R.V.)

• Definition: A real random variable is a mapping ( X : \Omega \rightarrow \mathbb{R} ) within a probability space ( (\Omega, \Theta, P) ).

• Pre-image: For any interval ( I \subseteq \mathbb{R} ), the pre-image ( X^{-1}(I) = { \omega \in \Omega : X(\omega) \in I } ) must belong to ( \Theta ).

• Set of Observables: The values taken by ( X ) are denoted as ( X(\Omega) ) and are called the set of observables.

• Remark: If ( \Theta = P(\Omega) ) (the power set), any mapping ( \Omega \rightarrow \mathbb{R} ) is considered a real random variable, which is common in discrete spaces.

2. Cumulative Distribution Function (CDF)

• Definition: The CDF ( F ) of a random variable ( X ) is given by:

  • ( F(x) = P(X \leq x) = P({ \omega \in \Omega : X(\omega) \leq x }) ).

• Description: Specifies the probability that the random variable ( X ) is less than or equal to ( x ) for any real value ( x ).

3. Discrete Real Random Variable (D.R.V.)

• Definition: A real random variable ( X ) is discrete if the set ( X(\Omega) ) is countable (either finite or countably infinite).

• Example: Tossing 3 fair coins:

  • Let ( Y ) be the number of heads:

    • ( P(Y = 0) = \frac{1}{8} )

    • ( P(Y = 1) = \frac{3}{8} )

    • ( P(Y = 2) = \frac{3}{8} )

    • ( P(Y = 3) = \frac{1}{8} )

4. Probability Mass Function (PMF)

• Definition: For a discrete random variable ( X ), the PMF is defined as ( P_X(x) = P(X = x) ) for ( x \in X(\Omega) ).

• Total Probability: ( \sum_{x \in X(\Omega)} P_X(x) = 1 ).

• Properties: If ( a ) is a value ( X ) never takes, then ( P_X(a) = 0 ).

5. Properties of CDF

1. Non-decreasing step function.

2. Constant in intervals: Value of ( F_X ) is constant in ([x_{i-1}, x_i)) and jumps by ( P_X(x_i) ) at ( x_i ).

3. Probability between intervals: ( P(a < X \leq b) = F_X(b) - F_X(a). )

4. Probability greater than a: ( P(X > a) = 1 - F_X(a). )

5. Limits: ( \lim_{x \to -\infty} F_X(x) = 0 ) and ( \lim_{x \to +\infty} F_X(x) = 1. )

• Remark: The CDF is uniquely determined by the PMF; conversely, the PMF can be derived from the CDF.

• Example of PMF and CDF:

  • PMF values: ( P_X(1) = \frac{1}{4}, P_X(2) = \frac{1}{2}, P_X(3) = \frac{1}{8}, P_X(4) = \frac{1}{8} )

  • CDF:

    • ( F_X(x) = 0, x < 1 )

    • ( F_X(x) = \frac{1}{4}, 1 \leq x < 2 )

    • ( F_X(x) = \frac{3}{4}, 2 \leq x < 3 )

    • ( F_X(x) = \frac{7}{8}, 3 \leq x < 4 )

    • ( F_X(x) = 1, x \geq 4 )

6. Continuous Random Variable

• Definition: A real random variable ( X ) is continuous if there is a continuous function ( f ) (the probability density function, PDF) such that:

  • ( P(a \leq X \leq b) = \int_{a}^{b} f(x) dx. )

• Properties of PDF:

  • ( f(x) \geq 0 ) for all ( x \in \mathbb{R} )

  • ( \int_{-\infty}^{\infty} f(x) dx = 1. )

  • So, ( P(X = a) = 0 ).

• Example:

  • If ( f(x) = egin{cases} C(4x - 2x^2) & 0 < x < 2 \ 0 & \text{otherwise} \ \ ext{Find } C ext{ such that } \int_{-\infty}^{\infty} f(x) dx = 1. \\ \ )

• Continuous CDF:

  • ( F_X(x) = P(X \leq x) = \int_{-\infty}^{x} f(t)dt. )

7. Expectation

8. Moments

9. Variance and Standard Deviation

• Variance: The variance of random variable ( X ), denoted ( \text{Var}(X) ), is:

  • ( \text{Var}(X) = E[(X - E[X])^2] = \mu_2(X) )

• Alternative Variance Calculation: ( \text{Var}(X) = E[X^2] - (E[X])^2 )

• Standard Deviation: Denoted ( \sigma(X) ), calculated: ( \sigma(X) = \sqrt{\text{Var}(X)}. )

10. Moment Generating Function (MGF)

• Definition: Moment generating function ( M_X(t) ).

• **Properties: }

  • For independent variables, ( M_{X}(t) ) equals the product of individual MGFs.

  • Uniquely determines the distribution of ( X ).

11. Distribution of a Function of a Random Variable

• Discrete case: Given ( Y = g(X) ), the PMF is given by:

  • ( P_Y(y) = \sum_{x:g(x)=y} P_X(x) ) for ( y \in Y(\Omega) )

• Continuous case: For strictly monotonic functions:

  • Use transformation to find PDF of ( Y = g(X) ).

12. Common Probability Models

• Discrete Distributions: Bernoulli, Binomial, Poisson, and Geometric distributions.

• Continuous Distributions: Uniform, Exponential, Gamma, and Normal distributions.

- Bernoulli Random Variable ( X \sim B(p) )

- Binomial Random Variable ( X \sim B(n, p) )

- Poisson Random Variable ( X \sim P(\lambda) )

- Geometric Random Variable ( X \sim G(p) )

- Uniform Random Variable ( X \sim U[a,b] )

- Exponential Random Variable ( X \sim E(\lambda) )

- Gamma Random Variable ( X \sim \gamma(a, b) )

- Normal Random Variable ( X \sim N(\mu, \sigma^2) )

Additional Examples and Remarks

• Consider number of successes in trials, defective products, etc.