Hypothesis testing

What are the 4 steps of any hypothesis test?

1) State H0 and H1. 2) Calculate the test statistic. 3) Find the critical value at your chosen alpha (or the p-value). 4) Compare, decide (reject / fail to reject H0) and conclude in words.

What is the null hypothesis (H0)?

The 'nothing is going on' claim you assume is true and try to find evidence against (e.g. beta_j = 0, or homoskedasticity, or no autocorrelation).

What does alpha (significance level) mean?

The probability of rejecting H0 when it is actually true (Type I error). You choose it, usually 0.05 (5%). It is not in the R output.

What is a Type I error vs a Type II error?

Type I: rejecting a true H0 (false positive; probability = alpha). Type II: failing to reject a false H0 (false negative).

What does the p-value mean?

The probability of getting a test statistic at least as extreme as the one observed IF H0 were true. It is NOT the probability that H0 is true.

p-value decision rule (applies to every test)

Reject H0 if p < alpha. Fail to reject H0 if p >= alpha.

What does 'fail to reject H0' mean?

The evidence was not strong enough to reject H0. It does NOT prove H0 is true.

Degrees of freedom for a regression (n - k - 1): what are n and k?

n = number of observations. k = number of explanatory variables NOT counting the intercept. n - k - 1 = observations minus number of coefficients estimated.

Where do I find n - k - 1 in R output?

In summary(): 'Residual standard error: ... on X degrees of freedom'. X = n - k - 1.

Which tests use only the upper (right) tail?

F-tests and chi-squared (LM) tests. Their statistics are always positive, so you only compare to the upper critical value. No two-sided version.

What is the difference between a coefficient test and a diagnostic test?

Coefficient tests (t, F): is a variable (or group) significant? Rejecting H0 = significant (usually the result you want). Diagnostic tests (BP, White, BG): are OLS error assumptions OK? Rejecting H0 = a problem with the model (bad news).

Why do heteroskedasticity / autocorrelation matter?

OLS coefficients are still unbiased, but the standard errors are wrong, so t-tests and F-tests can't be trusted. Fix: robust (White/HC) or HAC (Newey-West) standard errors.

t-test: what question does it answer?

Is ONE coefficient significantly different from a hypothesised value (usually zero)? i.e. does this variable have an effect on y?

t-test: hypotheses (two-sided)

H0: beta_j = 0 vs H1: beta_j not equal to 0. (One-sided: H1: beta_j > 0 or beta_j < 0.)

t-test: test statistic formula

t = (b_j - beta_0) / se(b_j), where b_j = estimated coefficient, beta_0 = hypothesised value (usually 0), se(b_j) = standard error of the estimate.

t-test: in words, what does t measure?

How many standard errors the estimate is away from the hypothesised value. Larger |t| = stronger evidence against H0.

t-test: distribution and degrees of freedom

t distribution with n - k - 1 degrees of freedom (under H0).

t-test: decision rule (two-sided)

Reject H0 if |t| > critical value t(alpha/2, n-k-1). Equivalent: reject if p < alpha.

t-test: decision rule (one-sided)

H1: beta > 0: reject if t > t(alpha). H1: beta < 0: reject if t < -t(alpha). Check the sign of t matches H1.

t-test: where to find b_j, se, t and p in R output?

Coefficient table: 'Estimate' = b_j, 'Std. Error' = se(b_j), 't value' = t (for H0: beta = 0), 'Pr(>|t|)' = two-sided p-value.

t-test: what if beta_0 is not zero?

R's 't value' only tests beta = 0. Calculate by hand: t = (Estimate - hypothesised value) / Std. Error.

t-table: how to find the critical value for a two-sided 5% test

Row = df (n - k - 1). Column = 0.025 one-tail (same as 0.05 two-tail). Check whether your table labels columns one-tail or two-tail.

t-table: how to find the critical value for a one-sided 5% test

Row = df (n - k - 1). Column = 0.05 one-tail.

t-table: what if my exact df isn't listed?

Use the next SMALLER df listed (conservative choice).

Large-sample t critical values (df = infinity row)

5% two-sided: 1.96. 5% one-sided: 1.645. 1% two-sided: 2.576. Rule of thumb: |t| > 2 is significant at about 5%.

F-test: what question does it answer?

Are several coefficients JOINTLY zero? i.e. do these variables, taken together, help explain y? (Also used for overall significance of the regression.)

Why use an F-test instead of several t-tests?

Running multiple t-tests compounds the false-positive risk, and individual t-tests can miss joint significance (e.g. correlated variables). F tests the group at once.

F-test: hypotheses

H0: the q coefficients being tested are all zero (e.g. beta_2 = beta_3 = 0). H1: at least one of them is not zero.

F-test: idea behind it

Impose the restriction (drop the variables) and see how much worse the fit gets. A big deterioration relative to normal noise means the variables matter, so reject H0.

F-test: SSR form of the statistic

F = [(SSR_R - SSR_UR) / q] / [SSR_UR / (n - k - 1)]

F-test: R-squared form of the statistic

F = [(R2_UR - R2_R) / q] / [(1 - R2_UR) / (n - k - 1)] (only valid when both models have the same dependent variable)

F-test: what is SSR?

Sum of squared residuals: add up every (actual y - fitted y) squared. Measures how much the model fails to explain. Smaller = better fit.

F-test: unrestricted vs restricted model

Unrestricted (UR) = full model with all variables, gives SSR_UR. Restricted (R) = model with the tested variables removed (coefficients forced to zero), gives SSR_R. SSR_R >= SSR_UR always.

F-test: what is q?

The number of restrictions = the number of coefficients you are testing equal to zero. Numerator degrees of freedom.

F-test: distribution and degrees of freedom

F distribution with (q, n - k - 1) degrees of freedom. Numerator df = q. Denominator df = n - k - 1 (from the UNRESTRICTED model).

F-test: decision rule

Reject H0 if F > critical value F(alpha; q, n-k-1). Upper-tailed only. Equivalent: reject if p < alpha.

F-table: how to find the critical value

Pick the table for your alpha (separate page for 5%, 1%, etc). Column = numerator df (q). Row = denominator df (n - k - 1). Where they meet = critical value. If df not listed, use next smaller.

F-test: how to get SSR from R output

SSR = (Residual standard error)^2 x (residual degrees of freedom). Read both from summary() of each model.

F-test: where is the overall F-test in R output?

Bottom line of summary(): 'F-statistic: F on q and df DF, p-value: ...'. H0: all slope coefficients = 0, so q = k.

F-test: relationship to the t-test

For a single restriction (q = 1), F = t^2.

t distribution: shape and parameter

Symmetric, bell-shaped, centred at 0, fatter tails than normal. One parameter: df. As df gets large it approaches the standard normal.

F distribution: shape and parameters

Always positive, right-skewed. Two parameters: numerator df (q) and denominator df (n - k - 1).

Chi-squared distribution: shape and parameter

Always positive, right-skewed. One parameter: df. Used for LM (n x R-squared) tests.

Which distribution does each test use?

t-test: t(n-k-1). F-test: F(q, n-k-1). Breusch-Pagan, White, Breusch-Godfrey (LM versions): chi-squared.

Which tests are two-sided vs upper-tailed only?

t-test: can be one-sided or two-sided. F, chi-squared (BP, White, BG): upper-tailed only.

Chi-squared 5% critical values, df = 1 to 5

df 1: 3.84. df 2: 5.99. df 3: 7.81. df 4: 9.49. df 5: 11.07.

Chi-squared table: how to find the critical value

Row = df. Column = RIGHT-tail probability = alpha (use 0.05 column for a 5% test).

What is an auxiliary regression?

A second, helper regression run only to detect a pattern in the residuals of your original model. You care about its R-squared, not its coefficients.

What is the 'normal' (original) regression in BP, White and BG?

Your original model estimated by OLS (y on the x's). You save its residuals (u-hat) and use them to build the auxiliary regression.

LM test statistic (general form)

LM = n x R-squared from the auxiliary regression. Under H0 it follows a chi-squared distribution.

Why does a HIGH R-squared in an auxiliary regression signal a problem?

It means the helper regression can explain a pattern in the residuals, i.e. the residuals are not random. That is evidence against H0 (homoskedasticity / no autocorrelation).

Heteroskedasticity tests: why square the residuals?

You care about the SIZE of the errors regardless of sign, so the dependent variable in the aux regression is u-hat squared.

Autocorrelation test: why NOT square the residuals?

You care about direction/sign (does a positive error tend to follow a positive error?), so the dependent variable in the aux regression is u-hat (not squared).

If I am only given aux regression output, how do I get LM?

LM = n x Multiple R-squared from that output. n = residual df + number of coefficients (rows in the coefficient table).

Breusch-Pagan (BP) test: what does it test?

Heteroskedasticity: does the error variance change systematically (linearly) with the regressors?

BP: hypotheses

H0: homoskedasticity (constant error variance). H1: error variance depends on the regressors (heteroskedasticity).

BP: normal (original) regression

Original model y on x1...xk by OLS. Save the residuals u-hat.

BP: auxiliary regression

u-hat squared on the original x1...xk (x's only, no squares or cross-products). Include a constant.

BP: test statistic

LM = n x R-squared from the auxiliary regression. (F-version: joint F-test that all slope coefficients in the aux regression = 0.)

BP: distribution and degrees of freedom

Chi-squared with k degrees of freedom (k = number of regressors in the aux regression, excluding the constant).

BP: decision rule and conclusion

Reject H0 if LM > chi-squared critical value (alpha, k) or p < alpha. Rejecting = heteroskedasticity present; use robust standard errors.

BP: what kind of heteroskedasticity can it detect?

Only variance that changes LINEARLY with the x's (it misses curved or interaction patterns; White's test covers those).

White test: what does it test?

Heteroskedasticity of a more general form than BP: error variance related to the x's, their squares and their cross-products.

White (standard): hypotheses

H0: homoskedasticity. H1: heteroskedasticity of some form.

White (standard): normal (original) regression

Original model y on x1...xk by OLS. Save the residuals u-hat.

White (standard): auxiliary regression

u-hat squared on the x's, the SQUARES of the x's, and the CROSS-PRODUCTS of the x's. Example with 2 x's: u2 = d0 + d1x1 + d2x2 + d3x1^2 + d4x2^2 + d5x1x2 + v.

White (standard): test statistic and df

LM = n x R-squared from the aux regression. Chi-squared with df = number of regressors in the aux regression (excluding constant). With 2 x's: df = 5. In general k(k+3)/2.

White (standard): decision rule

Reject H0 if LM > chi-squared critical value or p < alpha. Rejecting = heteroskedasticity; use robust standard errors.

White (special case): auxiliary regression

u-hat squared on y-hat and y-hat squared: u2 = d0 + d1(y-hat) + d2(y-hat^2) + v. (y-hat = fitted values from the original model.)

White (special case): test statistic and df

LM = n x R-squared from the aux regression. Chi-squared with 2 degrees of freedom, however many x's the model has.

White: why does the y-hat version work?

y-hat is a weighted sum of the x's, so y-hat squared expands into squares and cross-products of the x's. It is the standard White test with restrictions imposed (coefficients tied to the estimated slopes).

White: standard vs special case, when to use which?

Standard: more general, but df grows fast (uses up degrees of freedom). Special (y-hat): always 2 df, more power if variance rises with the predicted level of y, but can miss patterns tied to individual x's. Use special with many regressors or small samples.

White vs BP: what is the difference in the aux regression?

BP: u-hat squared on the x's only (linear). White: u-hat squared on x's PLUS squares and cross-products (or y-hat and y-hat squared in the special case).

Breusch-Godfrey (BG) test: what does it test?

Autocorrelation (serial correlation): are the errors related to previous errors? Mainly time-series data.

BG: hypotheses

H0: no autocorrelation up to order p (rho_1 = ... = rho_p = 0). H1: at least one rho is not zero.

BG: normal (original) regression

Original model y on x1...xk by OLS. Save the residuals u-hat.

BG: auxiliary regression

u-hat (NOT squared) on the original x's PLUS p lagged residuals (u-hat t-1, ..., u-hat t-p).

BG: why are the original x's in the aux regression?

To soak up any relationship between the residuals and the x's, so only the lagged-residual effect remains. The null concerns the rho coefficients on the lagged residuals only.

BG: test statistic

LM = (n - p) x R-squared from the aux regression (some texts / R's bgtest use n x R-squared). F-version: joint F-test on the lagged residuals.

BG: distribution and degrees of freedom

Chi-squared with p degrees of freedom (p = number of lagged residuals = order of autocorrelation tested).

BG: decision rule and conclusion

Reject H0 if LM > chi-squared critical value (alpha, p) or p-value < alpha. Rejecting = autocorrelation present; use HAC (Newey-West) standard errors or respecify (add lags).

BG vs Durbin-Watson: why prefer BG?

DW only tests first-order autocorrelation and is invalid when the model has a lagged dependent variable. BG tests higher orders and works with lagged dependent variables.

Compare the dependent variable in the BP, White and BG aux regressions

BP: u-hat squared. White: u-hat squared. BG: u-hat (not squared).

Compare the regressors in the BP, White and BG aux regressions

BP: original x's. White: x's + squares + cross-products (or y-hat and y-hat squared). BG: original x's + p lagged residuals.

Compare the chi-squared df for BP, White and BG

BP: k (number of x's). White: number of aux regressors excluding constant (2 for the y-hat version). BG: p (number of lagged residuals).

AR(p) model: definition

Autoregressive model: y_t depends on its own past values. yt = c + phi1*y(t-1) + ... + phip*y(t-p) + ut. p = number of lags of y.

AR(1) stationarity condition

|phi1| < 1. If phi1 = 1 the series is a random walk (unit root): shocks never fade and t/F tests are unreliable.

ARDL(p, q) model: definition

Autoregressive distributed lag: y_t depends on p lags of y AND the current x plus q lags of x. yt = c + phi1*y(t-1)+...+phip*y(t-p) + b0*xt + b1*x(t-1)+...+bq*x(t-q) + ut.

ARDL: in ARDL(p, q), what are p and q?

p = number of lags of the dependent variable y. q = number of lags of the explanatory variable x. (Check your course notes in case the order is reversed.)

ARDL: short-run and long-run effect of x on y

Short-run (impact) effect = b0. Long-run effect = (b0 + b1 + ... + bq) / (1 - phi1 - ... - phip).

AR / ARDL: how to choose lag lengths

Pick the (p, q) with the lowest AIC or BIC; or keep adding lags while the last lag is significant (t or joint F-test); check residuals show no autocorrelation (BG test).

Why is BG used (not Durbin-Watson) for AR / ARDL models?

These models include a lagged dependent variable as a regressor, which makes Durbin-Watson invalid. BG remains valid.

Special cases of ARDL

ARDL(p, 0): AR model plus current x. ARDL(0, q): pure distributed lag model (no lagged y). No x terms at all: AR(p).