1/20
Vocabulary and formula review flashcards covering probability distributions, regression derivations, interaction terms, ridge regression, sample size determination, and regression diagnostics.
Joint distribution from a chain of conditionals
The probability of joint events computed as P(X1=a,X2=b)=P(X1=a)×P(X2=b given X1=a).
Backward OVB (slope of X2 on X1)
The slope of X2 on X1 obtained from two regression tables using the formula b2b1′−b1, where b1′ is the slope of Y on X1 alone, and b1 and b2 are the multiple regression coefficients.
Slope of X1 on X2
The regression slope calculated from the reverse slope using the relation (slope of X2 on X1)×var(X2)var(X1).
Estimator aN+daS+c unbiasedness condition
For S=∑Y with Y∼Bernoulli(q), this estimator is unbiased when q=dc.
Interaction coefficient J (from a plot)
The change in slope between groups calculated as (slope of X1=1 group)−(slope of X1=0 group), where X1 is a 0/1 indicator and X2 is on the horizontal axis.
Coefficient b1 (from an interaction plot)
The vertical gap between the regression lines of the X1=1 group and the X1=0 group evaluated at X2=0.
cov(Y,X) in simple linear regression
Evaluates to b1×var(X), because cov(b0,X)=0, cov(b1X,X)=b1var(X), and cov(ε,X)=0 due to error independence.
var(Y) in simple linear regression
Evaluates to b12var(X)+var(ε), derived because variances of independent terms add, var(b1X)=b12var(X), and the constant b0 drops out.
Sampling distribution of the slope (no intercept, mean-zero X)
β^=N×var(X)∑(XiYi), which has mean b1, variance N×var(X)σε2, and is approximately Normal(b1,N×var(X)σε2).
Ridge shrunk toward a number b (sum loss)
The solution minimizing ∑(Y−βX)2+λ2(β−b)2, given by βR=∑(X2)+λ2∑(XY)+λ2b, which weights the least squares slope by ∑(X2)+λ2∑(X2).
Ridge toward b with averaged loss
The estimator minimizing N1∑(Y−βX)2+λ(β−b)2, given by βR=cβOLS+(1−c)b, where c=var(X)+λvar(X).
Sample size for a 95% interval to exclude v
The minimum number of observations required such that the lower confidence bound exceeds v (when q>v): N>(q−v1.96q(1−q))2.
Interaction with a random X1 (variance condition)
When Y=(stuff without X1)+cX1+ε conditional on X2 and X3, the conditional variance is c2var(X1)+var(ε); if c depends on X2 or X3, constant variance fails.
Non-indicators of regression assumption violations
Model properties including low R2, wide confidence intervals, and high variance with low bias in cross-validation (which indicates overfitting rather than assumption violation).
Indicators of regression assumption violations
Diagnostic red flags consisting of residual plot patterns (hump, funnel, bands, outliers) or having a binary Y.
Uniform(a,b) distribution
A continuous probability distribution with mean equal to 2a+b and variance equal to 12(b−a)2.
Expected value from a joint table
The expected value of a function g(X,Y) calculated as E[g(X,Y)]=∑g(x,y)×P(x,y) summed over all table cells.
Simulated code summary statistics
For simulated data where Y∣X∼Normal(1+2X,1): mean(y)=1+2E[X], var(y)=4var(X)+1, and mean(y[x==2])=1+2(2)=5.
Reading regression from a scatter plot
Visual estimation rules where b0 is the height of the cloud at x=0, b1 is rise over run, and σε=2half the vertical thickness of the cloud.
Orthogonal features
Feature functions ϕ1 and ϕ2 that satisfy E[ϕ1(X)ϕ2(X)]=0.
Correlation between fitted coefficients
The relationship where positively correlated predictors X1 and X2 yield negatively correlated coefficient estimates β^1 and β^2 (and vice versa); large coefficients of opposite sign suggest correlated predictors.