ch 8 regression: making the best possible predictions

• In this chapter we will make the best possible prediction of one variable from the other, and is the subject of this chapter

• We accomplish the task of making optimal predictions of one variable from another by finding the equation of a line called…

• the regression line

The reason for the rather strange sounding name of regression line is a  historical accident.

• Recall that Francis galton invented the scattergram and did much of the early work on correlational techniques

• When Galton Was examining the relationship between the height of fathers and their sons, he noticed that the heights of the sons of the tallest fathers tended to also be above average, but not as much as their extremely tall fathers. That is, the heights of Sons of very tall fathers tended to “regress” or fall back toward the mean.

• Therefore he called the line that described this phenomenon as a..

• regression line

• he could have more simply called it a prediction line, because that is what it is

• . The regression line is the line that generates the best predictions of one variable from the other.

The regression line is the line that..

generates the best predictions of one variable from the other.

• To illustrate the regression line, consider the example from chapter 7 of a fast food company developing a test to predict the length of time employees will stay on the job.

• The data from this example are reproduced here in table 8-1 and figure 8-1 shows a scattergram based on the data. Remember that in chapter 7 we found that the correlation between the test scores and months on the job was r= .76. According to our interpretation chart (table 7-3),  this is a strong correlation  and should allow us  to make predictions of months on the job with few large errors. 

images of table 8-1 and figure 8-1

• In the scattergram in figure 8-1 The solid line passing through a set of points is the regression (or prediction) line.

• The vertical distance between each point and the regression line ( represented for one case by a dotted line) is the prediction error for that employee. 

The best possible regression line is the line that will …

minimize the sum of the squared prediction errors (around the line).

• So, here we have another version of a sum of squares. We could graphically estimate the regression line is figure 8-1 by attempting to draw a line through the middle of the points, but this estimate would be inexact and could be quite inaccurate.

• Instead, we would like to find the exact equation of the ideal regression line

What must we do to create the ideal regression(prediction line)

we must find the equation of the straight line that will produce the minimum sum of squared errors around itself.  this equation is called a least  squares solution ( it minimizes the sum of the squared errors).

• mathematically, finding this equation is a classical minimization problem requiring calculus

  •  fortunately, mathematicians love to do this sort of thing, and they have graciously solved our problem for us.

• We only have to employ their solution. However, before we can do this, we need to be familiar with the analytical geometry of straight line.

THE analytical geometry of a straight line:

• If we look at the graph of a straight line ( for example, consider the line show in figure 8-2), two characteristics of the line turn out to be critical.

•  First, the line has a slope, or certain amount of steepness. The slope can range from steep to flat. Figure 8-3 shows a set of lines with different slopes.

slope and Y-int:

• Slope is defined mathematically as the amount of change in Y  relative to the amount of change in X for any particular line:

  • Slope = change in Y/ change in X

• In math the greek letter delta (Δ) is often used to denote change, so we can rewrite our equation using this symbol:

  • slope = ΔY/ΔX

• Note that for line a and figure 8-3 a small change in X produces a large change in Y, Because the slope of line a is steep.

• On the other hand, for line B the same amount of change in X produces only a small change in Y, because the slope of line B is relatively flat 

• The second characteristic of a straight line is that at some point it crosses the y-axis. The value of y at the point where the line crosses the Y-axis (where x=0) is called the Y intercept.

• Figure 8-4 shows a set of lines all of which have the same slope, but different Y intercepts.

Slope is  defined mathematically as

• the amount of change in Y  relative to the amount of change in X for any particular line:

  • Slope = change in Y/change in X

in math the greek letter delta (Δ) is often used to denote

• change, so we can rewrite our equation using this symbol:

  • slope = ΔY/ΔX

What is the first characteristic of a straight line

•  has a slope, or certain amount of steepness. The slope can range from steep to flat. Figure 8-3 shows a set of lines with different slopes.

The second characteristic of a straight line is that at some point it….

• crosses the y-axis. The value of y at the point where the line crosses the Y-axis (where x=0) is called the Y intercept.

What is the Y intercept

The value of y at the point where the line crosses the Y-axis (where x=0)

• The combination of its slope and Y intercept uniquely defines any straight line.

  • That is, for any particular combination of slope and Y intercept there will be one straight line that is described by that combination. 

  • Once we know the slope and the Y intercept of a line,  we can write the equation of that line.

    •  the equation of any straight line is the following form:
      
      

• Y= slope (X) + Y intercept

• Or if we let b Represent the slope of and a represent the Y intercept, the equation can be written:

  • Y= bX + a

    • b = slope

    • a= Y intercept 

What does b represent in the

•  equation of any straight line

  • Y= bX + a

the slope

what does a represent

•  equation of any straight line

• Y= bX + a

the y intercept

• As an example, suppose that we have a line with the slope of 2 and an Y intercept of 1. 

  • This slope tells us that, if we graph the line, the value of Y will increase 2 units for everyone unit that X increases. Rememeber , that slope…

    •  slope = change in Y/ change in X = ΔY/ΔX 

how would this look with the values in their proper place

• And the Y  intercept of 1 tells us that the line will cross the Y axis where y= 1 

• Since the equation for any straight line is the form Y= slope (X) + Y intercept or Y=- bX + a, the equation for the line in our example is Y= 2x + 1 

FINDING THE EQUATION FOR THE REGRESSION LINE 

• Returning to our search for the best regression (or prediction) line, we need to find only the slope and Y intercept of our regression line to uniquely define it. 

• As mentioned earlier, the best regression line is the line that minimizes the sum of the squared prediction errors around it. 

• What is the first step

• So we need to find the slope and Y intercepts of the line that meets this requirement.

• It turns out we can find the slope and Y intercept of the regression line using many of the same quantities that we used in computing the correlation coefficient(ch 7). 

  • First, we find the slope using the formula:

    • Where ΣX is the sum of the X scores, ΣY is the sum of the Y scores, ΣX² is the sum of the squared scores, ΣXY is the sum of the products of X multiplied by Y for each XY pair of scores, and N is the number of pairs of scores. 

What does ΣX and ΣX² mean

the sum of X scores and the sum of the squared X scores

What does ΣXY mean

the sum of the products of X multiplied by Y for each XY pair of scores,

What does N mean

the  number of pairs of scores. 

The second step of finding the equation for the regression line:

Second, we find the Y intercept using formula:

• a= Y intercept = Y- b X

  • Where Y(with a line above it)  is the mean of the Y scores, b is the slope that we just found in the previous step, and X(with a line above) is the mean of the X scores 

  • The form of the equation for a regression line is nearly identical to the form of the equation for any straight line.

    • However, note that we use Y’

      • as the symbol representing the predicted value of Y in our regression equation to distinguish it from the actual value of Y (which will almost always be somewhat different). 

        • Thus, the equation for a regression line will be of the form:

        • Y’ = bX + a

          • Y’ = predicted value pf Y

          • b= slope 

          • a= y intercept 

    • Our final step in finding the regression line is to plug the values for the slope and Y intercept into this general equation for a regression line

What does this mean

the mean of the Y scores

What does this mean

mean of the X scores

What does Y’ represent?

represents the predicted value of Y in our regression equation

• to distinguish it from the actual value of Y (which will almost always be somewhat different).

the equation for a regression line will be of the form:

• Y’ = bX + a

  • Y’ = predicted value of Y

  • b= slope 

  • a= y intercept 

What is the final step to finding the equation for the regression line

to plug the values for the slope and Y intercept into this general equation for a regression line

FINDING THE EQUATION OF A REGRESSION LINE: AN EXAMPLE

• To illustrate the computations involved in finding the equation of as regression line, consider our ex data on a test for predicting how long fast food employees will remain on the job. Table 8-2 shows these data (reprodcued from table 8-1) in the form of a worksheet with some necessary quantities also provided

the steps:

1. caluate the slope of the regression line

2. calculate the Y-int, which requires that we plug in the value we just obtained for the slope (b=.55)

3. plug these values for the slope and Y int into the equation for the regression line

USING THE REGRESSION EQUATION TO MAKE PREDICTIONS

• So far, we have generated the equation for the regression line that will have a smaller sum of squared errors around it than any other possible straigh line would have. This equation can be written in several ways:

  • Y’ = .55X + 2.05 or Predicted value of Y= (.55)(value of X) + 2.05

• 
  

• Or, in our specific case relating test scores to how long fast food employees remain on the job:

  • Predicted number of months on the job= (.55)(test score) + 2.05

• We can look at the regression equation as a rule that specifies the relationship between test score and months on the job for any specific test score. 

  • This allows use to make the best possible prediction of how long a person with any possible test score will stay on the job.

  • As an ex, consider joe, an applicant who scored 6 on the test. What is our best possible prediction of how long joe will stay on the job? To answer this question we plug joes test ssore into our regression equation and solve for Y’(the predicted value of Y)

    • Predicted value of Y= .55(test score) + 2.05

      • Y’ = .55X + 2.05

      • Y’ = .55(6) + 2.05

      • Y’ = 3.30 + 2.05

      • Y’ = 5.35

    • Our best possible prediction is that joe will stay on the job for 5.35 months. The mean length of stay on the job is 4.80 months (see table 8-2). 

      • Therefore, we would prpbbably want to hire joe because he appears to be likely to stay on the job longer than average

    • If we look again at out example data (table 8-3) and the scattergram with the regression line (firgure 8-6) we see that the regression line is indeed an ideal fit to the data.

      • Note also that the prediction errors are rather small (as our correlational coefficient of r=.76 indicated they would be)

We can look at the regression equation as a rule that…

 specifies the relationship between test score and months on the job for any specific test score. 

• This allows use to make the best possible prediction of how long a person with any possible test score will stay on the job

an example of

• look at the regression equation as a rule that specifies the relationship between test score and months on the job for any specific test score. 

  • This allows use to make the best possible prediction of how long a person with any possible test score will stay on the job.

joe:

• consider joe, an applicant who scored 6 on the test. What is our best possible prediction of how long joe will stay on the job? To answer this question we plug joes test ssore into our regression equation and solve for Y’(the predicted value of Y)

  • Predicted value of Y= .55(test score) + 2.05

    • Y’ = .55X + 2.05

    • Y’ = .55(6) + 2.05

    • Y’ = 3.30 + 2.05

    • Y’ = 5.35

  • 
    

  • Our best possible prediction is that joe will stay on the job for 5.35 months. The mean length of stay on the job is 4.80 months (see table 8-2). 

    • Therefore, we would prpbbably want to hire joe because he appears to be likely to stay on the job longer than average

  • If we look again at out example data (table 8-3) and the scattergram with the regression line (firgure 8-6) we see that the regression line is indeed an ideal fit to the data.

    • Note also that the prediction errors are rather small (as our correlational coefficient of r=.76 indicated they would be)