Linear Regression and Correlation Analysis

Understanding Correlation

• Correlation Coefficient ($r$): A statistical measure that indicates the strength and direction of a linear relationship between two quantitative variables.

  • Example from Transcript: A correlation of $r = 0.92$ was found between wine ratings and the cost of wine at a local store.

    • Interpretation: This strong positive correlation ($r = +0.92$) suggests that, in general, as the cost of the wine went up, so did its rating. Conversely, cheaper wines tended to have lower ratings. This means a wine with low ratings is likely to be less expensive, while a wine with high ratings is likely to be more expensive.

  • Key Principle: Correlation does not imply causation. (e.g., higher cost does not cause higher ratings, nor do higher ratings cause higher cost in a direct causal sense related to the reviewer's psychology or the economic structure being described here).

    • It simply describes the observed relationship or association between the two variables. The statement "having to pay more caused the reviewer to give a higher rating" incorrectly implies causation.

The Line of Best Fit (Regression Line)

• Purpose: The regression line, also known as the line of best fit, is a straight line that best describes the linear relationship between the explanatory variable ($x$) and the response variable ($y$) in a scatter plot.

• Methodology: This line is determined by fitting a line that minimizes the sum of the squared vertical distances (residuals) from each data point to the line. The line is positioned to make this total distance of vertical deviations the least.

• Prediction: The line of best fit allows us to predict a response variable value ($ilde{y}$, read as "y-hat" or "predicted y") for a given explanatory variable value ($x$).

  • Equation Form: The general equation for a linear regression line is often represented as $ilde{y} = a + bx$. (Sometimes written as $ilde{y} = b<em>0 + b</em>1x$)

    • $ilde{y}$ represents the predicted $y$ value.

    • $x$ represents the independent or explanatory variable.

    • $a$ (or $b_0$) is the y-intercept, which is the predicted value of $y$ when $x=0$.

    • $b$ (or $b_1$) is the slope, which describes the estimated change in $y$ for a one-unit increase in $x$.

Residuals

• Definition: A residual is the difference between the actual observed value of the response variable ($y$) and the value predicted by the regression line ($ilde{y}$). It measures how far off the prediction from the regression line is from the actual outcome.

  • Formula: $\text{Residual} = y - ilde{y}$

    • $y$ = The actual (real) observed value of the response variable.

    • $ilde{y}$ = The predicted value of the response variable from the regression line.

• Interpretation:

  • A positive residual means the actual value was higher than the predicted value (the model underpredicted).

  • A negative residual means the actual value was lower than the predicted value (the model overpredicted).

  • A residual of zero means the actual value exactly matched the predicted value.

• Example: If a scatter plot shows a Pearson correlation of SPP and weight as $0.971$, and the regression equation (e.g., $1.1 + 0.764x$) is known, you can find the residual by applying the formula based on a specific $x$ and its corresponding actual $y$ value.

Graphing and Calculating the Regression Line

• Requirements for a Line: To graph or define a unique straight line, you need either two distinct points on the line or one point and the slope of the line.

• Calculator's Approach (Least-Squares Regression): While the exact calculation for the slope and intercept can be complex (involving sums of squares),

  • A calculator typically identifies the mean of the x-values ($\bar{x}$) and the mean of the y-values ($\bar{y}$). The point $( \bar{x}, \bar{y} )$ is always a point that lies on the least-squares regression line.

  • It then calculates the slope ($b$) using statistical formulas and combines this with $( \bar{x}, \bar{y} )$ to mathematically derive the full equation for the line of best fit. You do not typically have to perform these calculations manually, but it's important to understand the underlying process.

Interpreting the Slope of the Regression Line

• General Structure of Interpretation: "For each one-unit increase in the explanatory variable (x), the response variable (y) is predicted to increase or decrease by the amount of the slope in the units of the response variable."

• Steps for Interpretation:

  1. Identify the X-variable and its unit.

  2. Identify the Y-variable and its unit.

  3. Identify the slope ($b$) including its sign (positive for increase, negative for decrease).

  4. Construct the interpretive sentence following the general structure.

• Example 1 (Hypothetical Scenario):

  • Equation Structure: $\tilde{y} = 118.8 - 1.644 \times \text{latitude}$

  • X-variable: Latitude (let's assume units like degrees).

  • Slope: $-1.644$.

  • Interpretation: For each one-unit (e.g., one degree) increase in latitude, the response variable ($y$) is predicted to decrease by $1.644$ units.

• Example 2 (Swim Time and Overall Finish Time):

  • Equation Structure: $\tilde{\text{Finish Time}} = 122 + 1.56 \times \text{Swim Time}$ (Here, "Swim Time" is the x-variable, and "Overall Finish Time" is the y-variable.)

  • X-variable: Swim Time (in minutes).

  • Y-variable: Overall Finish Time (in minutes).

  • Slope: $+1.56$.

  • Interpretation: For every one minute increase in swim time, the overall finish time is predicted to increase by $1.56$ minutes. (This matches the correct interpretation described in the transcript for a multiple-choice question).

Applying Regression and Residuals in Problems

• Problem Scenario: An athlete completed a swim in $34$ minutes. The residual for their actual finish time was $11$ minutes. We need to find the athlete's actual finish time.

  • Given:

    • $x = \text{Swim Time} = 34 \text{ minutes}$

    • $\text{Residual} = 11 \text{ minutes}$

  • Recall the Regression Equation: $\tilde{\text{Finish Time}} = 122 + 1.56 \times \text{Swim Time}$

  • Step 1: Calculate the Predicted Finish Time ($\tilde{y}$) for $x=34$ minutes:

    • $\tilde{y} = 122 + 1.56 \times (34)$

    • $\tilde{y} = 122 + 53.04$

    • $\tilde{y} = 175.04 \text{ minutes}$

  • Step 2: Use the Residual Formula to Find Actual Finish Time ($y$):

    • $\text{Residual} = y - \tilde{y}$

    • $11 = y - 175.04$

    • To solve for $y$: $y = 11 + 175.04$

    • $y = 186.04 \text{ minutes}$

  • Conclusion: The athlete's actual finish time was $186.04$ minutes. The positive residual ($11$ minutes) indicates that their actual time ($186.04$ min) was $11$ minutes longer than what the regression model predicted ($175.04$ min); the model underpredicted their actual finish time.

Understanding Computer Output for Regression

• Identifying Coefficients: Computer outputs from statistical software for regression analysis typically provide coefficients for the intercept (often labeled 'Constant') and each explanatory variable.

  • Example: If an output shows a 'Constant' value and a coefficient for 'Age in years' ($x$ variable):

    • The value next to 'Constant' represents the y-intercept ($a$ or $b_0$).

    • The value next to 'Age in years' represents the slope ($b$ or $b_1$) associated with 'Age in years'.

  • The regression equation would be constructed as: $\tilde{\text{Response Variable}} = \text{Constant Value} + (\text{Coefficient for Age}) \times \text{Age in years}$

    • For example, if the output values lead to a negative slope for 'price advertised', it means as the explanatory variable increases, the price advertised is predicted to decrease.

• R-squared ($R^2$): This value, often expressed as a percentage, indicates the proportion of the total variation in the response variable ($y$) that can be explained by the linear relationship with the explanatory variable ($x$) included in the model.

  • Example: If $R^2 = 89.4\%$, it means that $89.4\%$ of the variability in the response variable ($y$) can be accounted for or explained by the regression model involving the explanatory variable ($x$). When used in calculations, this percentage is converted to a decimal: $R^2 = 0.894$.}