Correlation and Linear Regression

Data Organisation, Analysis and Variable Types

• Independent Variable ($X$): Predictor or explanatory variable that is known, controlled, or manipulated.
• Dependent Variable ($Y$): Variable being predicted or determined.
• Correlation: Statistical tool that measures and describes the relationship between two variables using two values ($X$ and $Y$) per observation.
• Scatterplot: Visual diagram used to check for patterns between pairs of data, though it does not quantify relationship strength.

Coefficient of Correlation ($r$)

• Definition: Statistical measure of the direction and strength of the linear relationship between two variables, ranging from $-1.00$ to $+1.00$.
• Direction of Relationship:
  • Positive Correlation: Both variables change in the same direction ($X$ increases as $Y$ increases; direct relationship).
  • Negative Correlation: Variables change in opposite directions ($X$ increases as $Y$ decreases; inverse relationship).
• Strength of Relationship (Absolute Value Range):
  • $0$ to $0.1$: No correlation
  • $0.1$ to $0.4$: Weak correlation
  • $0.4$ to $0.7$: Moderate correlation
  • More than $0.7$: Strong correlation
  • $!1.00$: Perfect correlation ($+1.00$ or $-1.00$)
  • $0$: No linear relationship

Key Considerations in Correlation Analysis

• Correlation vs. Causation: Correlation describes a relationship between variables but does not prove a cause-and-effect relationship.
• Range of Values: Data collected from a restricted range can alter the correlation; a wider range is required for accurate population description.
• Outliers: Extreme data points can dramatically alter correlation values (e.g., shifting $r$ from $0.08$ to $0.85$).

Coefficient of Determination ($r^2$)

• Definition: The squared value of the correlation coefficient ($r^2$), ranging from $0$ to $1$.
• Interpretation: Measures the proportion of variability in the dependent variable ($Y$) accounted for by variation in the independent variable ($X$).
• Predictability: An $r$ value of $+0.5$ yields $r^2 = 0.25$, indicating $25\%$ of the variability in $Y$ is explained by $X$.

Simple Linear Regression

• Definition: Statistical technique used to model the relationship between one independent variable ($X$) and one dependent variable ($Y$).
• Linear Regression Equation: $Y' = a + bX$
  • $Y'$: Predicted average value of $Y$ for any given $X$.
  • $a$: $Y$-intercept (estimated $Y$ value when $X = 0$).
  • $b$: Slope of the line (change in $Y$ per one unit change in $X$).
  • Least squares principle is used to obtain values for $a$ and b$.\n* **Application Limit:** Substituting X$$ values beyond the range of the dataset makes predictions unreliable.

Applications and the COPAI Framework

• Prediction: Uses systematic relationships between variables to estimate unknown outcomes.
• Validity and Reliability: Evaluates test validity through correlation direction and strength, and measures consistency across measurement sets.
• COPAI Process: Correlation analysis identifies and quantifies variable relationships during the analysis phase, while linear regression builds predictive models to inform practical business strategies.