chapter 9: regression

Regression

Chapter Objectives

  • By the end of this chapter, you should understand…

    • The concept of a linear equation

    • The regression line and prediction

    • Hypothesis testing with regression

    • The concept of multiple regression

Correlation

  • X and Y variables:

    • Both X and Y are treated as dependent variables.

    • Important note: correlation does not equal causation.

Fundamentals of Linear Regression

  • Identify two types of variables:

    • Predictor variable (X):

      • Also known as the independent variable.

      • Values are known and used to predict another variable.

    • Criterion variable (Y):

      • Also known as the dependent or outcome variable.

      • Variable with unknown values predicted based on X.

Line of Best Fit

  • How to define a "best fitting" straight line when multiple options exist?

    • A best fit line is defined as a line that minimizes prediction errors.

    • This line is identified through least-squares error solution, which calculates the smallest possible distance between the regression line and the observed data points.

Equation of a Line

  • The formula representing a line:

    • y=mx+by = mx + b

      • m: slope of the line, representing the change in Y for a unit change in X.

        • Slope calculation:

          • Slope (mm) can be expressed as:
            extSlope=racextrisechangeinyextrunchangeinxext{Slope} = rac{ ext{rise change in } y}{ ext{run change in } x}

      • b: y-intercept of the line, where the line crosses the Y-axis.

    • Range and domain considerations also apply.

Using Linear Regression to Answer Questions

  • Applications of linear regression include answering the following:

    • Is a pattern evident in a set of data points?

    • Which equation of a straight line can best describe this pattern?

    • Are the predictions made from this equation statistically significant?

Method of Least Squares: Finding Best Fit

  • Method of Least Squares: A statistical procedure for computing the slope (b) and y-intercept (a).

  • Steps involved in the method:

    • Step 1: Compute preliminary calculations.

    • Step 2: Calculate the slope (b).

    • Step 3: Calculate the y-intercept (a).

  • Final regression equation:

    • extPredictedY(ildeYext)=bX+aext{Predicted Y (} ilde{Y} ext{)} = bX + a

Example Case Study: Effectiveness of Therapy Sessions

  • A psychologist aims to predict therapy effectiveness by correlating therapy sessions (X) to symptoms expressed (Y).

  • Sample selected: Eight patients; symptoms recorded at the start.

Example Data Table




  • Therapy Sessions (X) vs. Symptoms (Y)


    # of Therapy Sessions

    # of Symptoms



    0

    15



    3

    25



    4

    26



    6

    35



    8

    40



    12

    12





    Step 1: Preliminary Calculations



    • Given calculation of means and sums for X and Y:

      • Individual deviations from means computed:

        • Mx=5.25M_x = 5.25, My=2.75M_y = 2.75

        • Deviations and their products calculated ((X−MX)(Y−MY)(X-M_X)(Y-M_Y)).

    Step 2: Calculate the Slope (b)

    • Slope (b) formula: b=racSSXYSSXb = rac{SS_{XY}}{SS_X}

      • Example calculation yielding:

      • extSlope(b)=−0.57ext{Slope (b)} = -0.57

    Step 3: Calculate Y-Intercept (a)

    • Formula for y-intercept (a):

      • a=My−bMxa = M_y - bM_x

      • Example calculation yielding:

      • extY−Intercept(a)=5.74ext{Y-Intercept (a)} = 5.74

    Final Regression Equation

    • Resultant line equation:

      • Y=−0.57X+5.74Y = -0.57X + 5.74

    Predicting Values of Y

    • Example predictions based on regression equation:

      • For 4 therapy sessions:

        • Y=−0.57(4)+5.74=3.46Y = -0.57(4) + 5.74 = 3.46

        • Expected symptoms: between 3 and 4.

      • For 9 therapy sessions:

        • Y=−0.57(9)+5.74=0.61Y = -0.57(9) + 5.74 = 0.61

        • Expected symptoms: between 0 and 1.

      • To predict no symptoms (Y=0):

        • 0=−0.57(X)+5.740 = -0.57(X) + 5.74

        • Thus X≈10X ≈ 10 therapy sessions recommended.

    Hypothesis Testing with Regression

    • Regression analysis can test hypotheses on whether predictor variables (X) predict criterion variable (Y) in the population.

    Regression and Residual Variance

    • Understanding variance types:

      • Regression Variation: Variance in Y related to changes in X.

      • Residual Variation: Variance in Y not accounted for by X.

      • Total Variance in Y: Sum of regression and residual variance.

    F Statistic

    • The F statistic is a comparison between two variances, specifically useful in regression analysis:

      • Numerator: df for regression

      • Denominator: df for residual

    • F distribution is positively skewed and primarily includes values >= 0.

    Hypothesis Testing Example: Stress Levels and Incivility

    • Formulate hypotheses:

      • H0: No relationship exists between stress levels (X) and incivility (Y).

      • H1: A relationship exists between stress levels (X) and incivility (Y).

    Analyzing Variance in Y

    • Measure regression (dependent on X) and residual variance.

    • Total (T) and regression (R) variances must equal Y variance.

    Decision Criteria

    • Significance level set at .05.

    • Critical F value calculated using degrees of freedom.

    Reporting Results in APA Format

    • Report example:

      • A regression analysis found F (1, 3) = 22.04, p < .05, R² = .88.

    Multiple Regression

    • Definition: Statistical method using two or more predictor variables to predict outcomes in a criterion variable.

      • Incorporates multiple independent variables into the regression equation.

      • Regression equation structure expands to include multiple slopes:

        • Yilde=b1X1+b2X2+…+aY ilde{} = b_1X_1 + b_2X_2 + … + a

    Conclusion

    • Regression analysis frameworks offer deep insights into psychological research, with multiple regression allowing for complex modeling of phenomena.

    • Understanding and ability to report results accurately are key in statistical practices within psychology and related fields.


Regression is like trying to draw a straight line on a piece of paper that has a lot of dots on it. Imagine each dot shows how many cookies your friends have and how happy they are. We want to find the best straight line that helps us guess how happy your friends might be if they have a different number of cookies.

What is Regression?

  • When we talk about regression, we are looking at how two things are related, like cookies and happiness!

Important Words:

  • Predictor Variable: This is like the number of cookies.

  • Criterion Variable: This is like the happiness level of your friends.

Drawing a Line:

  • The straight line we draw is called the "line of best fit." It's like finding the best path through the dots.

  • This line helps us guess how happy someone might be if they have more cookies.

Making Predictions:

  • If you tell me a number of cookies, I can use our line to guess how happy your friend will be!

Finding Our Line:

  • To find our line, we use some math to find out where the line should go so it matches the dots as closely as possible.

Extra Fun: Testing Ideas

  • Sometimes, we want to see if our guesses are correct. We can check if our line really helps us predict happiness by looking at different friends and their cookies.

By understanding regression, we can figure out how different things, like cookies and happiness, are connected!