Testing Predictors and Evaluating Linear Models

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/23

flashcard set

Earn XP

Description and Tags

Interpreting significance tests for beta coefficients, calculating and interpreting R² and adjusted R² as a measure of model quality, locating information on the significance of individual predictors and overall model fit in R lm model output

24 Terms

1
New cards

What does evaluating a model consist of? When should this evaluation occur?

  • Evaluating individual coefficients

  • Evaluating overall model quality

  • Evaluating model assumptions


Consider these before accepting a set of results

2
New cards

What are the steps in hypothesis testing?

  1. RQ

  2. Stats hypothesis

  3. Define null

  4. Calculate an estimate of effect of interest

  5. Calculate appropriate test stat (+ the p-value associated with this)

  6. Evaluate test stat against the null


3
New cards

What should a research question (statement of what we intend to study) define?

  • Constructs under study

  • The association being tested

  • Direction of association

  • Target populations etc


4
New cards

What is a statistical hypothesis?

Testable mathematical statement

  • Null → precis, specifically states that the effect of interest = 0

    • Can only ever test this hypothesis → p-value = probability of observing test stat or more extreme value, assuming H0 is true

  • Alternative → non-specific, says effect of interest ≠ 0 (makes no claim about what it might really be equal to)


5
New cards

What is beta?

Gives the predicted change in y for a unit change in x

  • If x and y are unrelated → (unit) change in x won’t result in any change to the predicted value of y


6
New cards

What test statistic is associated with beta coefficients?

t-statistic

<p>t-statistic</p>
7
New cards

What is the standard error?

Measures how variable the sample is

  • Smaller SE = more precise estimate = good


8
New cards

What is the code for working out the t-statistic in R?

knowt flashcard image
9
New cards

What distribution does the t-statistic follow when the null hypothesis is true? What does this distribution vary according to?

t-distribution → varies depending on df (related to sample size + number of predictors in linear regression)

<p>t-distribution → varies depending on df (related to sample size + number of predictors in linear regression)</p>
10
New cards

What is alpha?

Probability of incorrectly rejecting the null hypothesis, even when it’s true

  • E.g. 0.05 alpha (2-tailed) = 5% risk of incorrectly rejecting null


11
New cards

When are we more likely to find a statistically significant effect?

When residuals are small + we have a large sample

12
New cards

What do critical values (t*) establish? How does this relate to alpha?

Boundary for significance

  • The probability that a t-statistic will fall within these extreme regions of the distribution given H0 is true = alpha


<p>Boundary for significance</p><ul><li><p>The probability that a t-statistic will fall within these extreme regions of the distribution given H0 is true = alpha</p></li></ul><p></p>
13
New cards

What is the code for finding the critical values in R?

knowt flashcard image
14
New cards

t-distribution and critical values example

knowt flashcard image
15
New cards

What will occur when we measure an outcome variable y? Why does this happen?

Some variability in that outcome

  • Hopefully because an IV, x, impacts y

  • Other variability arises because there is random variability in the world that can’t be accounted for/our model can’t explain


16
New cards

How do we quantify how good our model is? What calculations are associated with this?

Based on what proportion of the variability in our data it can explain

  • Coefficient of determination = proportion of variability accounted for by the predictors/that the model explains = R²

    • More variance accounted for = better fit model

      • E.g. 0.6695 = 66.95% of the variation in y is accounted for by the predictors

    • Represents the extent to which the prediction of y is improved when predictions are based on the linear relation between x and Y, compared to not considering x


<p>Based on what proportion of the variability in our data it can explain</p><ul><li><p><strong>Coefficient of determination </strong>= proportion of variability accounted for by the predictors/that the model explains = R²</p><ul><li><p>More variance accounted for = better fit model</p><ul><li><p>E.g. 0.6695 = 66.95% of the variation in y is accounted for by the predictors</p></li></ul></li><li><p>Represents the extent to which the prediction of y is improved when predictions are based on the linear relation between x and Y, compared to not considering x</p></li></ul></li></ul><p></p>
17
New cards

What is the total sum of squares? How is it calculated?

Distance from the mean → for every observation, find the difference between the observed value + the mean of all observed y values → square difference → add up all the squares

<p>Distance from the mean → for every observation, find the difference between the observed value + the mean of all observed y values → square difference → add up all the squares</p>
18
New cards

What is the residual sum of squares? How is it calculated?

Distance from model → for every observation, find the difference between that observation + the value predicted by the model → square the difference → add up all the square

<p>Distance from model → for every observation, find the difference between that observation + the value predicted by the model → square the difference → add up all the square</p>
19
New cards

What is the model sum of squares? How is it calculated?

Distance between model + mean → for every observation, find the difference between the predicted value for that observation + the mean of all observed y values → square difference → add them all up

  • If all the dots formed a line

    • Residual = 0

    • R^2 = 1

      • Sum of squares = sum squares total


<p>Distance between model + mean → for every observation, find the difference between the predicted value for that observation + the mean of all observed y values → square difference → add them all up</p><ul><li><p>If all the dots formed a line </p><ul><li><p>Residual = 0 </p></li><li><p>R^2 = 1 </p><ul><li><p>Sum of squares = sum squares total </p></li></ul></li></ul></li></ul><p></p>
20
New cards

What is R² equal to in R?

lm()’s multiple R-squared

<p>lm()’s multiple R-squared</p>
21
New cards

What is the calculation for an adjusted R²?

knowt flashcard image
22
New cards

Why is adjusted R² calculated?

  • 2 or more predictors → R² tends to be an inflated estimate of the corresponding population value

    • Biased estimate = estimate in stat procedure doesn't accurately correspond to population value

  • Due to random sampling fluctuation, even when R² = 0 in the population, its value in the sample may ≠ 0

    • In smaller samples, the fluctuations from 0 will be larger on average

    • With more predictors, there are more opportunities to add to the positive fluctuation

  • Adjusted R² gives a more conservative estimate


23
New cards

What does adjusted R² adjust for?

Sample size (n) + number of predictors (k)

24
New cards

Why might R² and adjusted R² be similar?

If the sample size is large + the number of predictors is small → if there were even more predictors, then adjusted R² would be a much more accurate rep of proportion varianc explained