Chapter 11: Regression Analysis in Social Sciences

In social sciences, which statistical tool is used to ascertain causal relationships between variables?

Regression analysis

One advantage of regression analysis is its ability to study the effects of multiple factors on a __ variable simultaneously.

Dependent

Regression analysis allows researchers to conduct estimation and __ at the same time.

Hypothesis testing

Which type of regression analysis is used specifically for interval level data?

Classical regression

Which type of statistical models are used to analyze categorical level data in a regression framework?

Generalized linear models

In the simple regression equation $Y = a + bX + e$, what does the symbol $Y$ represent?

The dependent variable

In the simple regression equation $Y = a + bX + e$, what does the symbol $X$ represent?

The independent variable

In a regression model, the symbol $a$ represents the __-intercept.

Y

Y-intercept ($a$)

The expected value of the dependent variable $Y$ when the independent variable $X$ is zero.

What is the statistical term for the symbol $b$ in the regression equation?

Slope (or regression coefficient)

What does the regression coefficient $b$ represent regarding the relationship between $X$ and $Y$?

The amount $Y$ is expected to change for each one-unit increase in $X$.

In the regression equation $Y = a + bX + e$, what does the symbol $e$ represent?

The error term (or disturbance term)

What is the conceptual implication of the existence of the error term ($e$) in a regression model?

It implies that $X$ cannot completely explain or determine $Y$.

The predicted relationship between an independent and dependent variable in simple regression is graphically presented as a __.

Straight line

What are the two primary tasks to be accomplished in a regression analysis?

Estimate regression coefficients and test their statistical significance.

In regression, what is the researcher trying to ascertain regarding the effect of $X$ on $Y$ in the population?

The direction and magnitude of the effect.

What does the abbreviation OLS stand for in the context of estimating regression coefficients?

Ordinary Least Squares

Besides OLS, what is the other major type of method used to estimate regression coefficients?

Maximum Likelihood (ML) method

Using the OLS method, what is the formula to calculate the intercept $a$?

$a = \bar{Y} - b\bar{X}$

According to the requirements for classical regression, at what measurement level should both variables ideally be?

Interval level

Classical regression assumes a __ relationship between the independent and dependent variables.

Linear

What assumption is made regarding the sampling method in classical regression analysis?

Random sampling

What distributional assumption is required for classical regression, especially with small samples?

Assumption of normality

Why is the Y-intercept ($a$) sometimes considered non-sensible in practical interpretation?

Because an $X$ value of zero may not be logical or exist within the data range.

Between the intercept ($a$) and the slope ($b$), which is considered more substantively important in regression analysis?

The slope ($b$)

If the regression coefficient $b$ is greater than zero ($b > 0$), what is the relationship between $X$ and $Y$?

They are positively related.

If the regression coefficient $b$ is less than zero ($b < 0$), what is the relationship between $X$ and $Y$?

They are negatively related.

A regression coefficient of $b = 0$ indicates that the independent and dependent variables are __.

Unrelated

The predicted value of $Y$ ($\hat{Y}$) is a function of which two components of the regression equation?

The intercept ($a$) and the slope ($b$)

What is the formula for calculating the prediction error (or disturbance) for a specific point?

$e = Y - \hat{Y}$

How is the Error Sum of Squares ($SS{error}$) calculated?

$SS{error} = \sum e^2 = \sum (Y - \hat{Y})^2$

Which sum of squares represents the total variation in $Y$ without knowing the effect of $X$?

Total Sum of Squares ($SS{total}$)

How is the Total Sum of Squares ($SS{total}$) calculated?

$SS{total} = \sum (Y - \bar{Y})^2$

Which sum of squares represents the variation in $Y$ that is explained by the independent variable $X$?

Regression Sum of Squares ($SS{reg}$)

What is the formula for $SS{reg}$ based on the difference between sums of squares?

$SS{reg} = SS{total} - SS{error}$

The ability of a regression line to make predictions is expressed as the __ in error.

Proportionate reduction

What does the PRE statistic show in regression analysis?

How much of the variance in $Y$ can be explained by $X$.

What is the formula for the Proportionate Reduction in Error (PRE)?

$PRE = \frac{SS{total} - SS{error}}{SS{total}}$

What is the alternative name for the PRE statistic in regression analysis?

Coefficient of determination ($r^2$)

The coefficient of determination measures the proportion of __ in $Y$ determined by $X$.

Variance

In bivariate regression, the coefficient of determination ($r^2$) is equal to the square of which correlation statistic?

Pearson's $r$

What is the numerical range for the value of the coefficient of determination ($r^2$)?

Between 0 and 1

What is the term for the quantity represented by $1 - r^2$?

Coefficient of non-determination

How is the Pearson correlation $r$ mathematically related to the regression slope $b$?

$r = b(\frac{s_x}{s_y})$

What is the primary difference between ANOVA and regression regarding the independent variable?

In ANOVA the IV is categorical; in regression it is interval level.

In regression ANOVA, the equivalent of 'between-group SS' is the variation between the __ values and the grand mean.

Predicted $Y$ (or $\hat{Y}$)

What is the formula to calculate $SS{reg}$ using the coefficient of determination?

$SS{reg} = r^2 SS{total}$

What is the formula to calculate $SS{error}$ using the coefficient of determination?

$SS{error} = (1 - r^2) SS{total}$

What ratio is calculated to test the statistical significance of a regression model?

$F$ ratio (or $F = \frac{MS{reg}}{MS{error}}$)

Mean Square Regression ($MS{reg}$)

$MS{reg} = \frac{SS{reg}}{df{reg}}$

Mean Square Error ($MS{error}$)

$MS{error} = \frac{SS{error}}{df{error}}$

How many independent variables are present in a simple regression model?

One

How many independent variables are present in a multiple regression model?

Two or more

What is the primary advantage of multiple regression over simple regression?

It allows control for the effects of more than one independent variable simultaneously.

What is the predictive formula for a multiple regression with two independent variables ($X$ and $Z$)?

$\hat{Y} = b_0 + b_1X + b_2Z$

What statistic provides an overall measure of fit for a multiple regression model?

Multiple coefficient of determination ($R^2$)

In multiple regression, what does each individual regression coefficient represent?

The net effect of that variable after controlling for all other independent variables.

Predictors in a regression model must be interval level, but __ variables can be used if their points are reasonably equidistant.

Ordinal

What special type of variable is used to include nominal predictors in a regression model?

Dummy variables

How are dummy variables coded numerically?

Dichotomously as 0 and 1.

Interaction Effect

The joint effect of two or more independent variables on the dependent variable, in addition to their main effects.

How can multiple regression test for interaction effects?

By including the product of the independent variables in the model.

Why is linear regression unsuitable for dependent variables that are dichotomous (0 or 1)?

Because the outcome is constrained to be either 0 or 1, violating the continuous interval assumption.

Which regression approach is preferred for predicting dichotomous dependent variables?

Logistic regression (or probit regression)

How are the 'odds' of an event defined mathematically?

The probability the event occurs divided by the probability it does not occur ($P / (1-P)$).

What is the numerical range of 'odds'?

From 0 to infinity.

Odds Ratio

The odds of an event for one group divided by the odds for another group.

Logistic regression models the __ of the odds of a dichotomous event.

Natural logarithm

What is the statistical symbol for the log-odds (logit)?

$L$

What is the formula for the log-odds (logit) $L$?

$L = \log(\frac{P}{1-P})$

What is the degree of freedom for the regression component ($df{reg}$) in a simple bivariate regression?

1

What is the degree of freedom for the error component ($df{error}$) in a simple bivariate regression with $N$ observations?

$N - 2$

The total sum of variations without knowing the effect of $X$ is represented by the formula $\sum (Y - \bar{Y})^2$, also known as __.

$SS{total}$

In the regression line equation, what determines if the relationship is positive or negative?

The sign of the slope $b$.

What does $SS{error}$ represent conceptually in the context of variation?

Variation in $Y$ that cannot be explained by $X$ (the effect of the error term).