Research methods: week 9: workshop: Power analysis
Question I
a) What is a type 1 error?
Type 1 error is a false positive, meaning resulted indicated rejected the null hypothesis and accepted the alternative hypothesis, but there actually was not a relationship between the variables
b) What is a type 2 error?
Type II error is a false negative, so the researcher concluded to retain the null hypothesis, however, there actually was a relationship between the variables
c) A social psychologist conducted a study with 25 participants who completed measures of extraversion and sensation seeking. He conducted a correlation analysis which was non-significant, r(23) = .37, p = .07. He concluded that there is no relationship between extraversion and sensation seeking. What type of error is the social psychologist likely to have made?
There is a small sample size? Therefore more likely for null hypothesis to be retained even if not the case, thus false negative and therefore a Type II error. Also doesn’t an r= 0.3 mean it is moderate?
d) A health psychologist examined the relationship between health locus of control and the number of fruits and vegetables eaten per day in a large sample of school children. She conducted a correlation analysis which was significant, r(2891) = .04, p = .03. She concluded that there is a positive relationship between children’s health locus of control and fruit and vegetable intake such that greater perceptions that health is under one’s own control is associated with eating more fruits and vegetables. What type of error is the health psychologist likely to have made?
The psychologists is likely to have made a type I error, which is false positive, the relationship between the vegetables eaten and the locus of control actually only explains 1.6% of total variance of the data.
e) In a survey of 60 senior managers, an occupational psychologist found a significant positive correlation between job satisfaction and length of time in the post, r(58) = .61, p < .001. How should the strength of this correlation (i.e. the effect size) be described?
strong (it is above 0.5% so total equal variance is 37.21%)
f) A significant positive correlation was also found between psychological wellbeing and length of time in the post, r(58) = .26, p = .04. How should the strength of this correlation (i.e. the effect size) be described?
Small/ weak (greater than 0.1)
g) A non-significant positive correlation was also found between conscientiousness and length of time in the post, r(58) = .09, p = .49. How should the strength of this correlation (i.e. the effect size) be described?
h) A significant positive correlation was also found between organisation commitment and length of time in the post, r(58) = .34, p = .01. How should the strength of this correlation (i.e. the effect size) be described?
i) Alpha is typically set at 0.05. What does alpha refer to?
the point at which the relationship between the variables is significant
j) Power is typically set at 0.80. What does power refer to?
the chance of finding the effect of the relationship in the sample size
k) What does statistical power depend on?
effect size
sample size
alpha size
Question II
A health psychologist developed an online intervention to help people to make plans to reduce their alcohol consumption. She wanted to test the intervention by randomly allocating drinkers to an intervention condition or a control condition. The dependent variable would be units of alcohol consumed in a typical week at six-month follow-up. She planned to test the effectiveness of the intervention using an independent t-test to compare the alcohol consumption of the two groups at follow-up. A meta-analysis on online health behaviour interventions conducted by Webb et al. (2010) estimated that online health behaviour interventions that included a planning task produced an average effect size of d = 0.25.
a) What type of power analysis should the health psychologist conduct to estimate the required sample size?
Type: A prior
b) Conduct a power analysis to estimate the required sample size, with alpha set at 0.05 and power set at 0.80. How many participants should the health psychologist recruit into each group on the basis of the power analysis?
We need 506 participants for a total sample size to find an effect size of d= 0.25. 253 in both condition
c) The health psychologist conducted her study, recruiting 338 participants, of whom 168 were randomly allocated to the intervention condition and 170 were randomly allocated to the control condition. What type of power analysis should the health psychologist conduct to estimate the minimum effect size that she would be able to detect in her experiment?
Sensitivity
d) What is the minimum effect size (d) that she would be able to detect, with alpha set at 0.05 and power set at 0.80?
The minimum effect size is 0.306 (3DP)
e) The health psychologist calculated that the effect size in her experiment was d = 0.28 (which was non-significant). What type of power analysis should the health psychologist conduct to determine the statistical power of the analysis she conducted?
Post-hoc
f) What level of power did the analysis achieve (with alpha set at 0.05)?
0.7279 (4SF)
g) What type of power analysis should the clinical psychologist conduct to determine the number of participants she would need to recruit to detect an effect size of d = 0.28?
Type: A priori
h) How many participants she would need to recruit to detect an effect size of d = 0.28, with alpha set at 0.05 and power at 0.80?
202 in the control condition and 202 in the intervention condition
Question III
A developmental psychologist is investigating the development of arithmetic fluency skills in a cross-sectional sample of pupils in Year 1 (n = 60), Year 2 (n = 60), and Year 3 (n = 60). She conducts a one-way ANOVA and obtains an effect size of f = 0.36. Now she wants to determine the smallest effect that she could have detected with high probability given her sample.
a) What type of power analysis should the developmental psychologist conduct?
Sensitivity
b) What is the minimum effect size (Cohen’s f) she could have detected, with alpha set at 0.05 and power at 0.80?
0.233