3-4. Correlation & Multiple Regression: Theory

Flashcards relating to Correlation and Multiple Regression models

What is correlation?

An association or dependency between two independently observed variables.

What does a Pearson correlation coefficient of 0.0 indicate?

That X and Y are completely independent of each other.

What does a Pearson correlation coefficient of 1.0 indicate?

That X and Y are identical to one another.

What does a Pearson correlation coefficient of -1.0 indicate?

That X and Y are exactly inverse to one another.

What does the regression coefficient represent?

The slope of the effect of one variable on the other.

What does the correlation coefficient represent?

The strength of the statistical relationship between variables.

What is the formula for calculating prediction error?

Y - Y' (actual value minus predicted value)

In the z-normalized case of regression, what is the regression coefficient?

The regression coefficient is identical to the correlation coefficient

What is the null hypothesis in regression analysis significance testing?

b=0, meaning there is no relationship between the variables.

What is partial correlation used for?

To measure the association between two variables (X,Y) after accounting for the effect of other variables (Z)

How is multicollinearity detected?

Multicollinearity is detected by finding high bivariate correlations (> 0.9) between predictors.

What is singularity?

Singularity refers to an entirely redundant variable.

What is Cook's distance used for?

Measuring the extremity of an outlier

What is homoscedasticity?

Homoscedasticity is when residuals stay relatively constant over the range of the predictor variable

What is heteroscedasticity?

Heteroscedasticity is when residuals vary systematically across the range of the predictor variable.

What are the rules of thumbs for the number of predictor variables, if medium effect size?

N > 50 + 8 * m

What kind of distribution is important for residuals in multiple regression analysis?

Residuals should be normally distributed

What does the F-ratio reflect?

The ratio of the explained variance against the residual variance.

What is the difference between Regression and Correlation?

Correlation expresses the reliability of relation of 2 vars, Regression allows prediction of the value of one based on other

What does Cohen’s f2 measure?

Effect size for a multiple linear regression can be estimated by Cohen’s f2