stats week 20 1020
Working with Expectations, Variances, and Independence
Course: ECO1020 Statistics for Economics
Instructor: Matthias Parey
Properties of Expectation, Variance, and Covariance
Key Properties:
Expectation of a Constant: E(a) = a
This property states that the expectation of a constant is the constant itself. It lays the foundation of expectation in probability theory.
Expectation of a Linear Combination: E(aX + b) = aE(X) + b
This allows us to compute the expectation of a linear transformation of a random variable X.
Expectation with Multiple Random Variables: E(aX + bY) = aE(X) + bE(Y)
Similarly, the expectation of a linear combination of two different random variables is the weighted sum of their individual expectations.
Variances and Covariances of Linear Combinations
Variance Properties:
Variance of a Constant: var(a) = 0
This showcases that a constant does not exhibit variability or dispersion.
Variance of a Linear Transformation: var(aX + b) = a²var(X)
Highlights how scaling a random variable affects its variance, while the addition of a constant does not.
Variance of Multiple Random Variables: var(aX + bY) = a²var(X) + b²var(Y) + 2ab cov(X, Y)
This formula accounts for the variances of individual variables and their covariance, indicating how they collectively impact variability.
Covariance Properties:
cov(X, X) = var(X): This means the covariance of a random variable with itself is its variance.
cov(X, Y) = E(XY) − E(X)E(Y): This defines covariance in terms of expectations, measuring how two variables vary together.
Independence of Random Variables
When two random variables X and Y are independent:
Multiplicative Expectation: E(XY) = E(X)E(Y)
This indicates that the expectation of the product of two independent variables is equal to the product of their expectations.
Variance of the Sum: var(X + Y) = var(X) + var(Y)
This property confirms that the variances add when variables are independent.
Zero Covariance: cov(X, Y) = 0
While zero covariance indicates no linear relationship, it is crucial to note that zero covariance does not imply statistical independence among the variables.
Conditional Moments
Conditional Mean:
Defined as E(Y | X = x)
This represents the expected value of Y given that the variable X equals a specific value. It is important for understanding relationships between variables.
Example: E(wages | gender = female) vs E(wages | gender = male). This highlights differences in expected wages based on gender.
Correlation Note:
If E(Y | X) = µY, then σXY = ρXY = 0: Under these conditions, the correlation between Y and X is zero, marking a lack of linear dependence.
Conditional Variance:
Defined as var(Y | X = x)
This captures how the variance of Y changes when conditioned on a specific value of X, serving as a crucial component in regression analysis.
Example: var(wages | gender = female) examines wage variability among females.
Conditional Independence
Two random variables X1 and X2 are conditionally independent if their joint distribution given a third variable Y satisfies:
f X1X2|Y(x1, x2 | y) = fX1|Y(x1 | y) fX2|Y(x2 | y)
This concept is important for multi-dimensional probability distributions and helps simplify analysis by reducing complexity.
Example of Conditional Independence
Scenario: Analyze an alarm system at a house and responses from neighbors, John (XJ) and Anna (XA). Neither neighbor communicates with each other.
Although XJ and XA are not independent (John calling influences Anna’s probability of calling), they are conditionally independent given Y (the state of the alarm). This highlights the distinction between independence and conditional independence in statistics.
Normal and Standard Normal Distributions
Normal Distribution:
Denoted as N(µ, σ²) where:
E(X) = µ: This is the mean of the distribution, representing the central tendency.
var(X) = σ²: This signifies the variance, illustrating the spread of the dataset.
Characteristics: Symmetric, skewness = 0, kurtosis = 3. A defining feature that characterizes numerous natural phenomena.
Example: Height distribution by gender serves as a practical illustration of normal distribution.
Standard Normal Distribution
Denoted as Z: Z ∼ N(0, 1)
The cumulative distribution function (CDF) is denoted as Φ(z), where Pr(Z ≤ z) = Φ(z).
Standardizing a Normal Distribution:
Symmetry property: Φ(z) = 1 − Φ(−z)
If X ∼ N(µ, σ²), then Z = (X − µ) / σ ∼ N(0, 1): This converts a normal variable into a standard normal variable.
Computing Probabilities in Normal Distribution
Given Y ∼ N(µ, σ²) and Z = (Y − µ) / σ:
Calculate:
Pr(Y ≤ c2) = Pr(Z ≤ (c2 − µ) / σ) = Φ((c2 − µ) / σ)
Pr(Y ≥ c1) = 1 − Pr(Z ≤ (c1 − µ) / σ) = 1 − Φ((c1 − µ) / σ)
Pr(c1 ≤ Y ≤ c2) = Φ((c2 − µ) / σ) − Φ((c1 − µ) / σ)
Example of Probabilistic Computation:
For adult male height, normally distributed with Mean = 70 inches (178 cm), Standard deviation = 4 inches (7.6 cm).
Z calculation for height of 65 inches:
Z = (65 - 70) / 4 = -1.25
Probability that Z ≤ -1.25 = Φ(-1.25) = 10.56%.
Bivariate and Multivariate Normal Distribution
Definitions:
Bivariate normal distribution: Refers to the joint distribution of two normally distributed variables.
Multivariate normal distribution: This encompasses the joint distribution of more than two normally distributed variables.
Properties:
For bivariate normal variables X, Y with covariance σXY,
aX + bY is distributed as N(aµX + bµY, a²σ²X + b²σ²Y + 2abσXY): This illustrates how linear combinations of bivariate normals remain normally distributed.
Marginal distributions of each variable within a multivariate normal distribution remain normal, a property that provides a foundation for various statistical analyses.
Zero covariances indicate independence among variables in multivariate normal distribution, ensuring that knowledge about one variable does not provide information about the other.
The conditional expectation of Y given X is linear: E(Y | X = x) = a + bx, portraying the relationship between the variables under linear assumptions.
Describing Binary Outcomes: The Bernoulli Distribution
Bernoulli Random Variable:
Defined for binary outcomes (0 or 1).
Probability Distribution for a biased coin:
Y = 1 (success) with probability p
Y = 0 (failure) with probability (1 − p)
Expectations and Variance of Bernoulli Variable
Expectation:
E(Y) = p: This indicates the average outcome expected from this binary scenario.
Variance:
var(Y) = p(1 − p): Illustrates the uncertainty in the outcome, with variance peaking when p = 0.5.
Binomial Distribution
Sum of several independent Bernoulli trials under the same probability of success (p).
Denoted as Y ∼ B(n, p): This shows the distribution of n independent Bernoulli trials.
Expectations and Variance:
E(Y) = n × p: Representing the expected number of successes in n trials.
var(Y) = n × p × (1 − p): This details how variability evolves with successive trials.
Binomial Distribution: The PDF
For a random variable r under binomial distribution:
r ∼ B(n, p): States the model for the random variable.
Probability of obtaining r successes: Pr(r) = nCr * p^r * (1 − p)^(n − r)
Here, Cr (n choose r) = n! / (r!(n − r)!) expresses combinations of outcomes and underpins the total probability distribution.