Regression, Probability, and Statistical Inference Review

0.0(0)
Studied by 1 person
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/20

flashcard set

Earn XP

Description and Tags

Vocabulary and formula review flashcards covering probability distributions, regression derivations, interaction terms, ridge regression, sample size determination, and regression diagnostics.

21 Terms

1
New cards

Joint distribution from a chain of conditionals

The probability of joint events computed as P(X1=a,X2=b)=P(X1=a)×P(X2=b given X1=a)P(X_1=a, X_2=b) = P(X_1=a) \times P(X_2=b \text{ given } X_1=a).

2
New cards

Backward OVB (slope of X2X_2 on X1X_1)

The slope of X2X_2 on X1X_1 obtained from two regression tables using the formula b1′−b1b2\frac{b_1' - b_1}{b_2}, where b1′b_1' is the slope of YY on X1X_1 alone, and b1b_1 and b2b_2 are the multiple regression coefficients.

3
New cards

Slope of X1X_1 on X2X_2

The regression slope calculated from the reverse slope using the relation (slope of X2 on X1)×var⁡(X1)var⁡(X2)(\text{slope of } X_2 \text{ on } X_1) \times \frac{\operatorname{var}(X_1)}{\operatorname{var}(X_2)}.

4
New cards

Estimator aS+caN+d\frac{aS + c}{aN + d} unbiasedness condition

For S=∑YS = \sum Y with Y∼Bernoulli⁡(q)Y \sim \operatorname{Bernoulli}(q), this estimator is unbiased when q=cdq = \frac{c}{d}.

5
New cards

Interaction coefficient JJ (from a plot)

The change in slope between groups calculated as (slope of X1=1 group)−(slope of X1=0 group)(\text{slope of } X_1=1 \text{ group}) - (\text{slope of } X_1=0 \text{ group}), where X1X_1 is a 0/10/1 indicator and X2X_2 is on the horizontal axis.

6
New cards

Coefficient b1b_1 (from an interaction plot)

The vertical gap between the regression lines of the X1=1X_1=1 group and the X1=0X_1=0 group evaluated at X2=0X_2 = 0.

7
New cards

cov⁡(Y,X)\operatorname{cov}(Y, X) in simple linear regression

Evaluates to b1×var⁡(X)b_1 \times \operatorname{var}(X), because cov⁡(b0,X)=0\operatorname{cov}(b_0, X) = 0, cov⁡(b1X,X)=b1var⁡(X)\operatorname{cov}(b_1 X, X) = b_1 \operatorname{var}(X), and cov⁡(ε,X)=0\operatorname{cov}(\varepsilon, X) = 0 due to error independence.

8
New cards

var⁡(Y)\operatorname{var}(Y) in simple linear regression

Evaluates to b12var⁡(X)+var⁡(ε)b_1^2 \operatorname{var}(X) + \operatorname{var}(\varepsilon), derived because variances of independent terms add, var⁡(b1X)=b12var⁡(X)\operatorname{var}(b_1 X) = b_1^2 \operatorname{var}(X), and the constant b0b_0 drops out.

9
New cards

Sampling distribution of the slope (no intercept, mean-zero XX)

β^=∑(XiYi)N×var⁡^(X)\hat{\beta} = \frac{\sum (X_i Y_i)}{N \times \widehat{\operatorname{var}}(X)}, which has mean b1b_1, variance σε2N×var⁡^(X)\frac{\sigma_{\varepsilon}^2}{N \times \widehat{\operatorname{var}}(X)}, and is approximately Normal⁡(b1,σε2N×var⁡^(X))\operatorname{Normal}\left(b_1, \frac{\sigma_{\varepsilon}^2}{N \times \widehat{\operatorname{var}}(X)}\right).

10
New cards

Ridge shrunk toward a number bb (sum loss)

The solution minimizing ∑(Y−βX)2+λ2(β−b)2\sum (Y - \beta X)^2 + \lambda^2 (\beta - b)^2, given by βR=∑(XY)+λ2b∑(X2)+λ2\beta_R = \frac{\sum (XY) + \lambda^2 b}{\sum (X^2) + \lambda^2}, which weights the least squares slope by ∑(X2)∑(X2)+λ2\frac{\sum (X^2)}{\sum (X^2) + \lambda^2}.

11
New cards

Ridge toward bb with averaged loss

The estimator minimizing 1N∑(Y−βX)2+λ(β−b)2\frac{1}{N} \sum (Y - \beta X)^2 + \lambda (\beta - b)^2, given by βR=cβOLS+(1−c)b\beta_R = c \beta_{\text{OLS}} + (1 - c)b, where c=var⁡(X)var⁡(X)+λc = \frac{\operatorname{var}(X)}{\operatorname{var}(X) + \lambda}.

12
New cards

Sample size for a 95% interval to exclude vv

The minimum number of observations required such that the lower confidence bound exceeds vv (when q>vq > v): N>(1.96q(1−q)q−v)2N > \left(\frac{1.96 \sqrt{q(1-q)}}{q - v}\right)^2.

13
New cards

Interaction with a random X1X_1 (variance condition)

When Y=(stuff without X1)+cX1+εY = (\text{stuff without } X_1) + c X_1 + \varepsilon conditional on X2X_2 and X3X_3, the conditional variance is c2var⁡(X1)+var⁡(ε)c^2 \operatorname{var}(X_1) + \operatorname{var}(\varepsilon); if cc depends on X2X_2 or X3X_3, constant variance fails.

14
New cards

Non-indicators of regression assumption violations

Model properties including low R2R^2, wide confidence intervals, and high variance with low bias in cross-validation (which indicates overfitting rather than assumption violation).

15
New cards

Indicators of regression assumption violations

Diagnostic red flags consisting of residual plot patterns (hump, funnel, bands, outliers) or having a binary YY.

16
New cards

Uniform⁡(a,b)\operatorname{Uniform}(a, b) distribution

A continuous probability distribution with mean equal to a+b2\frac{a+b}{2} and variance equal to (b−a)212\frac{(b-a)^2}{12}.

17
New cards

Expected value from a joint table

The expected value of a function g(X,Y)g(X, Y) calculated as E[g(X,Y)]=∑g(x,y)×P(x,y)E[g(X, Y)] = \sum g(x, y) \times P(x, y) summed over all table cells.

18
New cards

Simulated code summary statistics

For simulated data where Y∣X∼Normal⁡(1+2X,1)Y \mid X \sim \operatorname{Normal}(1 + 2X, 1): mean⁡(y)=1+2E[X]\operatorname{mean}(y) = 1 + 2 E[X], var⁡(y)=4var⁡(X)+1\operatorname{var}(y) = 4 \operatorname{var}(X) + 1, and mean⁡(y[x==2])=1+2(2)=5\operatorname{mean}(y[x==2]) = 1 + 2(2) = 5.

19
New cards

Reading regression from a scatter plot

Visual estimation rules where b0b_0 is the height of the cloud at x=0x=0, b1b_1 is rise over run, and σε=half the vertical thickness of the cloud2\sigma_{\varepsilon} = \frac{\text{half the vertical thickness of the cloud}}{2}.

20
New cards

Orthogonal features

Feature functions ϕ1\phi_1 and ϕ2\phi_2 that satisfy E[ϕ1(X)ϕ2(X)]=0E[\phi_1(X) \phi_2(X)] = 0.

21
New cards

Correlation between fitted coefficients

The relationship where positively correlated predictors X1X_1 and X2X_2 yield negatively correlated coefficient estimates β^1\hat{\beta}_1 and β^2\hat{\beta}_2 (and vice versa); large coefficients of opposite sign suggest correlated predictors.