STATS ex 2 - conducting a power analysis

Conducting a power analysis

1) state research problem and question

2) offer null and alternative hypothesis

3) a priori power analysis - hypothesize the effect size

• effect size information includes any values in the study that allow the reader to directly or indirectly calculate an effect size

Effect Size

indicates how strong the relationship is between variables and the strength of the differences between groups

the degree to which the phenomenon is present in the population or the degree to which the null hypothesis is false

Smaller vs Larger effect size

Larger: would be selected if the researcher thought that the effect would be larger

Smaller: requires larger samples to detect the small differences

if the power is low only consider conducting the study if the sample is large enough to detect an effect

types of analysis used to determine effect size

r, Cohen’s d, R², Cohen’s f, d, OR (odds ratio)

calculated as: small, moderate, and large

Cohen’s d

most common effect size measure for a two sample design (comparison of 2 groups)

d=(x1-x2)/SD

x1= mean of group 1 and x2 = mean of group 2

SD - standard deviation of either group considering that both groups are assumed to have approximately equal SDs

d - difference between means of g1 and g2 in SD units

1.0 = two groups differed by exactly 1 SD unit from one another

(it can be used when there are more than 2 groups but can only calculate one pair at a time)

Cohen’s d example question:

if the mean score for group 1 was 7(3) and the mean score for group 2 is 6.65(2.75) what is the Cohen’s d ?

(7-6.5)/3 =0.167 orrr (7-6.5)/2.75=0.182

Cohen’s f

expresses effect size in SD units but for two or more groups

used to determine differences between groups

identify the magnitude of the differences among all groups but it will not explain the differences between specific groups

2 groups - f=1/2d or d=2f

Pearson r

computed on two continuous, approximately normally distributed variables

represented by the letter r and the value is always between -1.00 and +1.00

Pearson r INDICATE

0 = ?

positive correlation? +

negative correlation? -

0 indicates no relationship between the two variables

( + ) correlation indicates higher values of x are associated with higher values of y

( - ) correlation indicates higher values of x are associated with lower values of y

R²

coefficient of determination - effect size for linear regression

this represents the percentage of variance explained in y by the predictor

Simple Linear Regression (R²)

provides a means to estimate the value of a dependent variable based on the value of an independent variable

Multiple Regression (R²)

an extension on simple linear regression - more than one independent variable is entered onto the analysis to predict a dependent variable

Odds Ratio - OR

• used to obtain and indication of association when both the predictor and the dependent variable are dichotomous

ratio of the odds of an event occurring in one group to the odds of it occurring in another group

simply a means of comparing whether the odds of a certain event is the same for two groups

• can be used when the dependent variable is dichotomous and the predictor is continuous but a logistic regression analysis is then used

Logistic regression (OR)

allows a predictor to be tested against the dependent variable and will determine the influence on your after adjusting for the presence of the other predictors

d

used for a two sample comparative design where the dependent variable is dichotomous (nominal y/n)

represents the different in percentages in g1 vs g2

example of d

employment rate in group 1 is 50% and in group 2 is 65% this the anticipated “d” is 65%-50% =0.15 15%

always big minus small

what are the four components of a power analysis?

alpha or level of significance (0.05)

standard power (.80 or 80%)

effect size (strength of relationship)

sample size

according to the table in the lecture what would be considered a moderate Cohen’s f value?

0.25

according the the table from lecture what would be considered a large Pearson r?

.50

Prior to performing the power analysis the research team is trying to decide between an a at 0.05 or an a of 0.01

which one will require a larger sample size?

setting the alpha to 0.01 will require more study participants

(a smaller alpha requires a larger sample size because the statistical test will require a larger critical statistical value to yield significance)

the research team is planning to compute their statistics using a One-Way ANOVA.

what effect size measure should be used in the power analysis ?

Cohen’s f

the research team is planning to compute their statistics using an independent samples t-test.

what effect size measure should be used in the power analysis

Cohen’s d

review of Cohen’s d

effect that represents the magnitude of the difference between two groups expressed in standard deviation units

ex - after calculating cohen’s d the following was determined 150 participants are needed (75 a group) to study a hypothesis using an independent samples t-test based upon Cohen’s d of 0.50, a=0.05 and power = 0.80

means: Cohen’s d = 0.50 is a moderate effect size

review of Cohen’s f

determines the difference between the groups when planning a one-way ANOVA

identifies the magnitude of the differences among the groups but it will not explain the differences between specific groups

ex - after calculating Cohen’s f the following was determined 969 participants are needed (323/group) to study a hypothesis using a One-Way ANOVA based upon Cohen’s f of 0.10, a=0.05, and power=0.80 what does this mean

cohen’s f = 0.10 means small effect size (large study group needed)

review of Pearson r

determines the correlation between two groups

ranges between -1.00 and +1.00 where 0 indicates no relationship

review of R²

used for linear regression

represents the percentages of variance explained in y by the predictor

review of Odds ratio

used when the dependent variable is dichotomous and the predictor is continuous or dichotomous

OR is a way of comparing whether the odds of a certain event is the same for two groups

computed by performing a logistic regression when testing a predictor (or set of predictors) with a dichotomous dependent variable

review of d

used when both predictor and dependent variable are dichotomous

preferred in a power analysis for the two-sample comparative design where the dependent variable is dichotomous

represents the difference in percentages in pairs of proportions

prior to conducting a power analysis a researcher reviewed the literature and discovered that a study similar to hers reported an OR value of 1.2. According to the table how would you characterize the magnitude of the effect ?

OR = 1.2 means small effect size, large group #

a researcher is planning to compute a Pearson r. What effect measure should be used in the power analysis ?

Pearson r is its own correlation measure (so OR)

Joe Smith is conducting a research study that seeks to determine if there is a difference in temperature in dormitory rooms based upon the residence hall. Joe randomly selects 10 rooms from residence hall A and 10 rooms from residence hall B. What statistical test should be used if the data was not normally distributed

Mann-Whitney U (alternative to ind. samples t-test because it is not normally distributed)

Sammy is conducting a study to determine if the level of satisfaction with cafeteria food is different between freshman, sophomore’s, juniors and seniors at Nebraska universities. To conduct this study Sammy used independent random samples from all the universities in Nebraska. What statistical analysis should Sammy use?

Kruskal-Wallis Test

(compares differences between 2+ groups and dependent variable is not interval/ratio scaled variable)