Linear Regression

• Define linear regression and its purpose in data analysis

• Explain assumptions to perform a linear regression

• Interpret regression coefficients

• Apply linear regression concepts to pharmacy related experiments

Learning Objectives

It described the degree of which the change in value of one variable corresponds to a change in the value of a second variable - measures the STRENGTH of association 

ex: One variable changes, and another changes also - How related are they? Does one cause the other? 

What is Correlation in terms of statistical analyses?

It is a statistical technique to estimate the relationship between a dependent variable (the response) and one or more independent variables (The explanatory) - It measures what the association IS

ex: AS X increases by 1 unit, how much does Y increase by? How much does X affect Y? 

What is regression in terms of statistical analyses? 

The Dependent variable (Y)

Regression analysis also allows us to predict ______ by fitting a linear equation to observed data

y = mx + b

What is the regression analysis equation?

• Y_i = β₀ + β₁X_i + E_i

• (i = 1, 2, 3…. n)

• β₀ = Y intercept

• β₁ = slope = average amount of change in the dependent variable (Y) for each one unit change in X (independent variable) 

• E_i = residual for i’th subject (Difference between what is observed (Yi) and what the model predicts 

What is the simple linear regression equation when you have multiple (“n”) pairs of measures

equal to zero

In regard to the simple linear regression equation, for the NULL HYPOTHESIS, the slope for the regression line would be:

NOT equal to zero

• Two measures ARE associated

• Use two-tailed tests to assess

In regard to the simple linear regression equation, for the ALTERNATIVE HYPOTHESIS, the slope for the regression line would be:

1. F test - asks “is there an association between X and Y?” 

   1. if R² is equal to 0 

   2. if P-value is <0.05 = evidence suggests measures ARE associated 

2. T-tests - asks “are the slope and intercept significantly different from 0?”

What is the two-phase-hypothesis test for seeing if two variables are associated? 

• Normality assumption for residuals

• Linear relationship between dependent and independent variables

• NO Heteroskedasticity present 

  • In simpler words:

    • In a perfect regression, the dots (errors) should be evenly spread around the line at all values of X.

    • Heteroskedasticity happens when the dots start fanning out or shrinking — meaning the variance of errors is not constant.

• Check for multicollinearity among independent variables - if highly correlational, deep them from model (measuring same thing)

• Screen for outliers in both independent and dependent variables

What are the assumptions for Simple linear regression?

The difference between prediction (expectation) and observation (Ei, etc)

• The predicted values are the ones gained from the regression equation

• SPSS can calculate this (Observed - predicted = residuals) 

What are residuals?

smaller ones

• Larger R² = larger coefficient of determination (ratio of predicted value variance to total variance) 

RESIDUALS NEED TO BE NORMALLY DISTRIBUTED IN ORDER OT PERFORM LINEAR REGRESSION

The best regression residuals are ____

Describes how much variability for this specific independent variable (DBP) is not explained by other independent variables in the model (Should be > 0.10) 

Tolerance definition - in regard to SPSS calculation of regression

A measure to detect and quantify multicollinearity (Should be <10)

Variance Inflation Factor definition - in regard to SPSS calculation of regression

• They are the probability-probability plot data points that should be close to the LINE

• Scatter plot data points fall into rectangle and have no standard deviation exceeding 3 or -3 along the X or Y axis

Residuals information - in regard to SPSS calculation of regression

Expressed in the actual units of each independent variable; these are the numbers we plug into the equation to predict the values of the independent variable

Unstandardized coefficients definition

Expressed as standardized value for all independent variables in the model so we can compare them against each other on the same scale

Standardized coefficients definition

There is a proportional relationship between X and Y

On the linear regression graph (After computing equation), if there is a POSITIVE, UPWARD slope, that means

There is an inversely proportional relationship between X and Y

On the linear regression graph (After computing equation), if there is a NEGATIVE, DOWNWARD slope, that means

Little to no relationship between X and Y

On the linear regression graph (After computing equation), if there is a POSITIVE, BUT SMALL, MINIMAL increase in slope, that means