RM2 Final

power analysis, multiple regression, dummy codes, effect codes, interactions with categorical and continuous variables

What are the different ways to do sample size planning

power analysis, accuracy in parameter estimation, sequential analysis

Describe power analysis

stems from hypothesis significance testing

Describe accuracy in parameter estimation

concerned with estimating effect sizes with a specific degree of precision

Describe sequential analysis

approach concerned with the efficiency of data collection. optimal with hard to reach populations or limited resources for data collection.

A correct conclusion happens under what two conditions

null hypothesis is true and you fail to reject the null hypothesis (nothing is happening and we say nothing is happening)

A type one error happens under what two conditions

null hypothesis is true but we reject the null hypothesis (nothing is happening but we say something is happening)

A type two error happens under what two conditions

null hypothesis is true and we fail to reject the null hypothesis (something is happening but we say that nothing is happening)

What is the other type of correct conclusion

null hypothesis is false and we reject the null hypothesis (something is happening and we say something is happening)

A type 1 error is also known as a

a false positive

A type 2 error is also known as a

false negative

the probability of making a type II error is also known as

Beta

Describe B

the risk level/probability of making a type one error

Describe power

the ability to find an effect that is really there

Power equation is

(1-Beta), where Beta is the probability of making a type 2 error

How can we increase Power

Power, Alpha/type 1 error rate, N/sample size, E/effect size

The effect size of f² is also known as

the signal to noise ratio

f² equation is

R²/1-R²

What is Cohen’s rule of thumb for effect sizes

small = 0.02, medium = 0.15, large = 0.35

One way to increase power in a study is to reduce noise by doing what

making MSE smaller

How to make MSE smaller

control extraneous variance in the measure, include relevant variables as predictors, don’t include highly correlated predictors in the model

We can include power by increasing what else

predictor variance, make sure your measure captures a wide range

What are the 3 analyses we can do when performing a power analyses

post-hoc analysis, a priori analysis, sensitivity analysis

describe a post-hoc analysis

try to avoid this since sample effect size is being used as the population effect to estimate power. sample size might be artificially inflated

describe an a priori analysis

requires lots of information. It finds out how many people you need in your study before the study is conducted. But you need to know the effect size you expect

describe a sensitivity analysis

finds out how much power you have given a certain sample size, effect size, and type 1 error. Useful for secondary data analysis or determining the level of power after the study is conducted

What are the 3 things you need to conduct an a priori analysis

alpha level, power level, and effect size

desired power level is often what

0.80, 0.90, 0.95

How can we estimate the desired effect size

pilot study, hunch, past research, aim for the smallest meaningful effect

What are the 4 levels of measurement

nominal, ordinal, interval, ratio

describe a ratio level of measurement

same as interval but theres an absolute zero

describe an interval level of measurement

rank ordered levels and intervals are consistent

describe an ordinal level of measurement

rank-ordered levels. can tell which values are higher or lower but intervals across the scale may not be consistent

Describe dummy coding

assign values of 1 or 0 to each code where 1 is the target for that group and zero is assigned to everything else

With dummy coding (no interaction), the intercept is what

the mean for the reference group

With dummy coding (no interaction), the coefficients represent

the mean level difference in the outcome between the reference group and the group coded as 1 for that dummy code

Dummy codes are not independent of each other because

if you are in one category you are unlikely to be in another, the predictors are correlated

The R² for the set of dummy codes (non interaction) provides what

an F-test, which is the same for the overall ANOVA

Describe effect coding

you assign values of 0, 1, -1, to each code such that 1 = target for that level, -1 = reference group, 0 for everything else

with effect coding (no interaction), the intercept means what

the grand mean of the outcome

with effect coding (no interaction), the coefs represent what

the coefs connected to the effect code represent the mean-level difference between the grand mean and the group coded as one for that effect code

With 2 level categorical predictors a positive b1 coef means what

mean for the target group (1) is larger than the mean for the reference group (-1)

Is there a test for the reference group in effect coding

no, you need to derive another set of effect codes

How do you interpret the intercept with dummy coding and 1 categorical + 1 continuous predictor

Or the mean value of the outcome for the reference group with a score of 0 on the control

How do you interpret the coefs with dummy coding and 1 categorical + 1 continuous predictor

predicted mean level difference in the outcome between the target and reference group, holding the continuous variable constant

How do you interpret the continuous variable with dummy coding and 1 categorical + 1 continuous predictor

predicted change in the outcome for every 1 unit increase in the continuous predictor, holding the categorical variable constant

How do you interpret the intercept variable with effect coding and 1 categorical + 1 continuous predictor

the average value of the outcome variable across the categorical variable that have a score of 0 on the continuous predictor

How do you interpret the coefs variable with effect coding and 1 categorical + 1 continuous predictor

the predicted mean-level difference in the outcome between the target and the grand mean, holding the continuous variable constant

How do you interpret the continuous variable with effect coding and 1 categorical + 1 continuous predictor

predicted change in the outcome for every 1 unit increase in the continuous variable, holding the categorical variable constant

How do you get the effect of the continuous variable on the outcome for the reference group with dummy coding

you plug in 0 for the other dummy codes in the regression equation

How do create an interaction term between the continuous and categorical variables

you need to multiply the continuous variable by each level of the categorical variable

How do you test the significance of an interaction term

use a hierarchical regression between a basic model and the interaction model and the F test will tell you whether the new interaction terms explain unique variance in the outcome over and above the first order effects

What does the intercept represent with the inclusion of an interaction, cont., and categorical variable for dummy coding

the intercept captures the predicted level/average of the outcome for the reference group

What does the first-order effect for the cont. predictor represent with the inclusion of an interaction, cont., and categorical variable for dummy coding

the simple slope for the effect of the cont. predictor on the outcome for the reference group. Is it NOT the main effect of the cont. predictor

What does the coef represent with the inclusion of an interaction, cont., and categorical variable for dummy coding

b1 (coef) predicted mean level difference in outcome between the target group and the reference group with avg. levels of the cont. predictor

What does the interaction coef represent with the inclusion of an interaction, cont., and categorical variable for dummy coding

b4 - predicted diff in the effect of the cont. predictor on the outcome between the target and the reference group. the the difference in the slope for the cont. predictor on the outcome for the target relative to the reference group.

To get the predicted outcome for the reference group with avg. levels of the cont. predictor with effect coding, what should you do

you must multiply each effect code by -1 and add this value to the intercept. Y = (b0 + b1+ b2+ b3) + b4*Income_Centered

To get the predicted outcome for the target group with avg. levels of the cont. predictor with effect coding, what should you do

add the coef to the intercept. Y = (b0 + b1) + b4*Income_C

How can you get the simple slopes with effect coding with an interaction term

you cant. You have to switch the effect codes to dummy codes

to get the simple slopes for an interaction model with a cont. and categorical predictor with effect codes

you have to switch out the effect codes with dummy codes and conduct multiple regression analyses to get the simple intercept and slope of cont. predictor on the outcome for each level of the categorical variable, each regression will change what the reference group is

What’s the simple slope for the reference group in an interaction model with a cont. and categorical predictor with effect codes

b0 - avg. outcome for the reference group and b4 (coef for the cont. predictor): the effect of cont. predictor for the reference group

What’s the simple slope for the other coefs/new reference group in an interaction model with a cont. and categorical predictor with effect codes

b0 - avg. outcome for the new reference group and b4 (coef for the cont. predictor): the effect of the cont. predictor for the reference group

when the effect code to look at cont. by categorical interactions what happens to the intercept and the 1st order effect of the cont. predictor

intercept is the average outcome for that reference group, the 1st order effect of the cont. predictor is the average effect of the cont. predictor