Study Notes on Correlation and Regression

Chapter 10: Correlation and Regression

Section 1: Scatterplots and Correlation

  • Introduction to Scatterplots

    • Scatterplots were previously discussed in Chapters 1 and 2.
    • Importance of correlation and regression.
    • Purpose of this section: understand scatter plots and compute correlation coefficients.

  • Drawing Scatterplots

    • Use ordered pairs to create scatter plots.
    • Practicing calculations and correlation coefficients.
    • Testing hypotheses about relationships between variables.

  • Variables in Statistical Analysis

    • Explanation of independent and dependent variables.
    • Independent Variable (Explanatory Variable): Can be manipulated or controlled (e.g., the number of hours studied is the independent variable).
    • Dependent Variable (Response Variable): The outcome that depends on the independent variable (e.g., the final grade).
    • Example: A teacher studies the relationship between hours of study and student grades.

  • Scatterplot Description

    • Each pair of values represents a point on the scatter plot.
    • Points can range from few to many in number, resulting in various shapes.
    • Scatter plots help visualize relationships:
    • Positive linear relationship: points trend upward.
    • Negative linear relationship: points trend downward.
    • Curvilinear relationship: non-linear shapes.
    • No relationship: random distribution of points.

  • Example Analysis of Relationships

    • Building Heights: Analyzing correlation between building height (x-axis) and stories (y-axis).
    • Observed positive linear relationship.
    • Absences and Grades: Exploring correlation between absences and final grades.
    • Visualization indicates a negative linear relationship.
    • Wind Speeds: No apparent correlation between average and maximum wind speeds.
    • Scatter plot appears random, indicating a lack of relationship.

  • Correlation Coefficient Overview

    • A numerical measure of the strength of the linear relationship between two variables.
    • Defined mathematically with the formula:
      r=n×∑(xy)−∑x×∑y(n×∑(x2)−(∑x)2)×(n×∑(y2)−(∑y)2)r = \frac{n \times \sum (xy) - \sum x \times \sum y}{\sqrt{(n \times \sum (x^2) - (\sum x)^2) \times (n \times \sum (y^2) - (\sum y)^2)}}
    • Types of Correlation Coefficients:
    • Population Correlation Coefficient (p): Used with the entire population.
    • Sample Correlation Coefficient (r): Used with a sample; main focus of current study.
    • Pearson Product-Moment Correlation Coefficient: Another name for sample correlation coefficient (r).

  • Ranges and Interpretations

    • The value of r ranges from -1 to 1:
    • Close to -1 indicates a strong negative relationship.
    • Close to 1 indicates a strong positive relationship.
    • A value near 0 indicates little to no linear correlation.
    • Example values:
      • r = -0.944 (strong negative), r = 0.927 (strong positive), r = 0.172 (weak positive).

  • Properties of Correlation Coefficient

    • Unitless measure; independent of the measurement units of variables.
    • Correlation coefficient is sensitive to outliers, which can dramatically affect r's value.

  • Assumptions for Correlation Coefficient Validity

    • Random sample.
    • Data pairs should approximately fall on a straight line measured at interval/ratio levels.
    • Variables should have bivariate normal distribution.

  • Calculating Correlation Coefficient: Example

    • Prepare a table with x, y, xy, x², and y² columns.
    • Populate values from dataset (e.g., absences and final grades) to compute totals.
    • Substitute into the correlation coefficient formula:
    • Example outcome: r=−0.944r = -0.944 indicating a strong negative correlation.

  • Hypothesis Testing for Correlation

    • Null Hypothesis (H0): p = 0 (no relationship).
    • Alternative Hypothesis (H1): p ≠ 0 (relationship exists).
    • Allowable methods for testing:
    1. T-test for significance.
    2. P-value calculation.
    3. Critical value method using correlation significance tables (Table A-8).

  • Correlation vs. Causation

    • Important distinction: correlation does not imply causation.
    • Five potential explanations for rejecting null hypotheses:
    1. Variable x causes changes in y.
    2. Variable y causes changes in x (swapped variables).
    3. A lurking variable drives the relationship.
    4. A complex relationship exists among variables.
    5. Coincidental evidence arises.

Section 2: Regression

  • Overview and significance of Regression Line

    • The regression line minimizes the sum of squared residuals.
    • Formula for the regression line: y′=a+bxy' = a + bx
    • aa: y-intercept, bb: slope, xx: independent variable, y′y': predicted value.

  • Understanding Residuals

    • Residual: difference between the actual value and the predicted value by the regression line.
    • The goal of regression is to minimize total residual distance, denoted as least squares.

  • Regression Coefficients Calculation

    • Formulas:
    • For y-intercept (a):
      a=∑(y)∑(x2)−∑(x)∑(xy)n∑(x2)−(∑(x))2a = \frac{\sum (y) \sum (x^2) - \sum (x) \sum (xy)}{n \sum (x^2) - (\sum (x))^2}
    • For slope (b):
      b=n∑(xy)−∑x∑yn∑(x2)−(∑x)2b = \frac{n \sum (xy) - \sum x \sum y}{n \sum (x^2) - (\sum x)^2}
    • Example trends and their regression line interpretations.

  • Predictions Using Regression Lines

    • Assumptions for valid predictions:
    • Scatter plot points approach the regression line.
    • Sample remains random.
    • r's significance confirmed.
    • Predictions can be made by substituting values into the regression equation.
    • Extrapolation warning: predicting beyond the dataset's x-range can lead to nonsensical results (e.g., predicting height when far beyond ranges).

  • Handling Outliers and Influential Points

    • Influential points can significantly sway the regression line's position.
    • Two approaches to handle outliers:
    1. Remove them from the analysis.
    2. Collect additional data near the outlier to better inform the regression.

  • Using Software (Excel) for Regression Calculations

    • Procedure for using Excel functions to compute correlation coefficients and regression lines utilizing the built-in functions (e.g., =CORREL(), =LINEST()).
    • Creating and interpreting scatter plots in Excel with trend lines vis-a-vis regression calculations.

  • Key Takeaways from the Lecture

    • Review of correlation computations and practical implications of regression analysis in statistics.
    • Continuous emphasis on systematic checking for errors using visual data representation.