stats week 20 1020

Working with Expectations, Variances, and Independence

Course: ECO1020 Statistics for Economics

Instructor: Matthias Parey

Properties of Expectation, Variance, and Covariance

Key Properties:

• Expectation of a Constant: E(a) = a

  • This property states that the expectation of a constant is the constant itself. It lays the foundation of expectation in probability theory.

• Expectation of a Linear Combination: E(aX + b) = aE(X) + b

  • This allows us to compute the expectation of a linear transformation of a random variable X.

• Expectation with Multiple Random Variables: E(aX + bY) = aE(X) + bE(Y)

  • Similarly, the expectation of a linear combination of two different random variables is the weighted sum of their individual expectations.

Variances and Covariances of Linear Combinations

Variance Properties:

• Variance of a Constant: var(a) = 0

  • This showcases that a constant does not exhibit variability or dispersion.

• Variance of a Linear Transformation: var(aX + b) = a²var(X)

  • Highlights how scaling a random variable affects its variance, while the addition of a constant does not.

• Variance of Multiple Random Variables: var(aX + bY) = a²var(X) + b²var(Y) + 2ab cov(X, Y)

  • This formula accounts for the variances of individual variables and their covariance, indicating how they collectively impact variability.

• Covariance Properties:

  • cov(X, X) = var(X): This means the covariance of a random variable with itself is its variance.

  • cov(X, Y) = E(XY) − E(X)E(Y): This defines covariance in terms of expectations, measuring how two variables vary together.

Independence of Random Variables

When two random variables X and Y are independent:

• Multiplicative Expectation: E(XY) = E(X)E(Y)

  • This indicates that the expectation of the product of two independent variables is equal to the product of their expectations.

• Variance of the Sum: var(X + Y) = var(X) + var(Y)

  • This property confirms that the variances add when variables are independent.

• Zero Covariance: cov(X, Y) = 0

  • While zero covariance indicates no linear relationship, it is crucial to note that zero covariance does not imply statistical independence among the variables.

Conditional Moments

Conditional Mean:

• Defined as E(Y | X = x)

  • This represents the expected value of Y given that the variable X equals a specific value. It is important for understanding relationships between variables.

  • Example: E(wages | gender = female) vs E(wages | gender = male). This highlights differences in expected wages based on gender.

Correlation Note:

• If E(Y | X) = µY, then σXY = ρXY = 0: Under these conditions, the correlation between Y and X is zero, marking a lack of linear dependence.

Conditional Variance:

• Defined as var(Y | X = x)

  • This captures how the variance of Y changes when conditioned on a specific value of X, serving as a crucial component in regression analysis.

  • Example: var(wages | gender = female) examines wage variability among females.

Conditional Independence

• Two random variables X1 and X2 are conditionally independent if their joint distribution given a third variable Y satisfies:

  • f X1X2|Y(x1, x2 | y) = fX1|Y(x1 | y) fX2|Y(x2 | y)

  • This concept is important for multi-dimensional probability distributions and helps simplify analysis by reducing complexity.

Example of Conditional Independence

• Scenario: Analyze an alarm system at a house and responses from neighbors, John (XJ) and Anna (XA). Neither neighbor communicates with each other.

  • Although XJ and XA are not independent (John calling influences Anna’s probability of calling), they are conditionally independent given Y (the state of the alarm). This highlights the distinction between independence and conditional independence in statistics.

Normal and Standard Normal Distributions

Normal Distribution:

• Denoted as N(µ, σ²) where:

  • E(X) = µ: This is the mean of the distribution, representing the central tendency.

  • var(X) = σ²: This signifies the variance, illustrating the spread of the dataset.

  • Characteristics: Symmetric, skewness = 0, kurtosis = 3. A defining feature that characterizes numerous natural phenomena.

  • Example: Height distribution by gender serves as a practical illustration of normal distribution.

Standard Normal Distribution

• Denoted as Z: Z ∼ N(0, 1)

  • The cumulative distribution function (CDF) is denoted as Φ(z), where Pr(Z ≤ z) = Φ(z).

  • Standardizing a Normal Distribution:

    • Symmetry property: Φ(z) = 1 − Φ(−z)

    • If X ∼ N(µ, σ²), then Z = (X − µ) / σ ∼ N(0, 1): This converts a normal variable into a standard normal variable.

Computing Probabilities in Normal Distribution

• Given Y ∼ N(µ, σ²) and Z = (Y − µ) / σ:

  • Calculate:

    • Pr(Y ≤ c2) = Pr(Z ≤ (c2 − µ) / σ) = Φ((c2 − µ) / σ)

    • Pr(Y ≥ c1) = 1 − Pr(Z ≤ (c1 − µ) / σ) = 1 − Φ((c1 − µ) / σ)

    • Pr(c1 ≤ Y ≤ c2) = Φ((c2 − µ) / σ) − Φ((c1 − µ) / σ)

• Example of Probabilistic Computation:

  • For adult male height, normally distributed with Mean = 70 inches (178 cm), Standard deviation = 4 inches (7.6 cm).

  • Z calculation for height of 65 inches:

    • Z = (65 - 70) / 4 = -1.25

  • Probability that Z ≤ -1.25 = Φ(-1.25) = 10.56%.

Bivariate and Multivariate Normal Distribution

Definitions:

• Bivariate normal distribution: Refers to the joint distribution of two normally distributed variables.

• Multivariate normal distribution: This encompasses the joint distribution of more than two normally distributed variables.

Properties:

• For bivariate normal variables X, Y with covariance σXY,

  • aX + bY is distributed as N(aµX + bµY, a²σ²X + b²σ²Y + 2abσXY): This illustrates how linear combinations of bivariate normals remain normally distributed.

• Marginal distributions of each variable within a multivariate normal distribution remain normal, a property that provides a foundation for various statistical analyses.

• Zero covariances indicate independence among variables in multivariate normal distribution, ensuring that knowledge about one variable does not provide information about the other.

• The conditional expectation of Y given X is linear: E(Y | X = x) = a + bx, portraying the relationship between the variables under linear assumptions.

Describing Binary Outcomes: The Bernoulli Distribution

Bernoulli Random Variable:

• Defined for binary outcomes (0 or 1).

• Probability Distribution for a biased coin:

  • Y = 1 (success) with probability p

  • Y = 0 (failure) with probability (1 − p)

Expectations and Variance of Bernoulli Variable

• Expectation:

  • E(Y) = p: This indicates the average outcome expected from this binary scenario.

• Variance:

  • var(Y) = p(1 − p): Illustrates the uncertainty in the outcome, with variance peaking when p = 0.5.

Binomial Distribution

• Sum of several independent Bernoulli trials under the same probability of success (p).

• Denoted as Y ∼ B(n, p): This shows the distribution of n independent Bernoulli trials.

Expectations and Variance:

• E(Y) = n × p: Representing the expected number of successes in n trials.

• var(Y) = n × p × (1 − p): This details how variability evolves with successive trials.

Binomial Distribution: The PDF

• For a random variable r under binomial distribution:

  • r ∼ B(n, p): States the model for the random variable.

• Probability of obtaining r successes: Pr(r) = nCr * p^r * (1 − p)^(n − r)

  • Here, Cr (n choose r) = n! / (r!(n − r)!) expresses combinations of outcomes and underpins the total probability distribution.