Research Methods- Results: Mapping Stats to Design

What must statistical procedures be appropriate to?

study design and data type

What is done if there is more than one appropriate statistical method for a given design to answer the research questions?

select one procedure and give the rationale as to why it was chosen

What does choosing the more powerful tool increase?

confidence in results

What does statistical analysis directly answer?

research question

Because statistical analyses directly answer research questions, researchers are allowed to...?

draw reasonable conclusions with some confidence

What do statistical analyses map directly onto?

study design

How should results be displayed?

clearly and unambiguously

What do results naturally lead into?

discussion that addresses research questions and hypotheses

What are used to visually represent groups' performance trends for each experimental task?

graphs

What do graphs provide?

great visual about overall performance, trends, and interactions

What type of graph is used for continuous data?

line graph

What type of graph is used for categorical data?

bar graph

What are tables used for?

to compare/contrast each group's performance on the experimental tasks

What are tables a good display of?

the entire range of performance

What do tables give specific details about?

how each group performed

What is typically displayed in a table?

mean, SD, range

What is the first step of mapping stats to design?

data organization

What kind of data structure is necessary?

one that will allow analyses to be run without error

What is the second step of mapping stats to data?

data analysis

Statistical analyses

statistical techniques/procedures that allow you to determine whether results are significant or nonsignificant

What is the heart of data analysis?

inferential statistics

What do inferential statistics allow?

ability to address research questions/hypotheses and make reasonable inferences from results

What is inferential stats applied to?

a limited set of data collected on a small sample representing the large population

Because inferential stats are based on the performance of samples, they allow...?

generalizability of results to large populations

What results are displayed on the output of data analysis?

1. Summary data (descriptive summary results)

2. Inferential data (results addressing research questions/hypotheses)

2a. parametric stats

2b. non-parametric stats

What is wise to do before running inferential stats?

inspect data by looking at summary descriptive stats

What does summary descriptive stats tell?

if there are data problems (non-normally distributed or skewed data)

Descriptive stats

any data set has its own distributional characteristics and properties represented by central tendency, variability, and skewness

Central tendeny

mean, median, mode

Median

middle score of distribution

Mode

most frequently occurring value

Variability

variance, standard deviation, range

Variance

mean of squared deviation

Standard deviation

average deviation from mean (usually what is reported in paper)

Range

highest to lowest value (usually reported in paper)

Skewness

bunching up of scores at one end or another on the normal curve

What may skewness be?

positive or negative

Skewness in negative direction

lower scores bunched up

Skewness in positive direction

higher scores bunched up

Bimodal distribution

two groups with two different distributions and little to no overlap

Multimodal distribution

different distributions for every group- everybody is behaving differently

What does it mean when there is a bimodal or multimodal distribution?

nature of the task is likely not reliable

What type of stats allow the researched to answer research questions/hypotheses, draw inferences from the data, and draw conclusions?

inferential statistics

What are the two types of inferential statistics?

parametric and non-parametric

When can parametric stats be used?

when the data meets the 4 assumptions

What are the 4 assumptions for the use of parametric stats?

data are interval or ratio

population parameter of interest is normally distributed

groups demonstrate comparable variance

sufficient n size per group

If distributions and/or variances assumptions are violated, what 3 options does the researcher have for data analysis?

1) transform non-normally distributed data to better normalize them

2) if data are interval or ratio, use linear effects modeling

3) use appropriate analogous non-parametric procedure

If data violate 1 or more parametric assumptions, then what?

run non-parametric stats

What should the author be clear about?

if the data have violated assumptions 2-4 leading them to use an appropriate non-parametric procedure

What is the third step of mapping stats to data?

interpreting the data and drawing inferences

What is the purpose of data interpretation?

to confidently make reasonable inferences and extend results from tested sample to larger population

What does generalizability of results mean?

good external validity

What are methods used by researchers to generalize results from samples to populations?

inferential statistics

What are inferential stats based on?

probability theory

Researches interpret results to draw one of what two conclusions?

groups DO NOT statistically differ from each other (null Ho) or groups DO statistically differ from each other (research Ho)

Which hypothesis are research studies set up to test?

null

Null Ho

no difference between groups (no statistical difference), meaning the groups come from the same population

Research Ho

there is a difference between groups (statistical significance) meaning the groups come from different populations

What does the researcher compare?

results of inferential statistics expressed as p-value for each obtained p result to .05 alpha level

What is p-value?

probability or significance level

What is the most liberal alpha level to accept as significant?

.05

What do all studies generate?

one or more "obtained" p-value

What does the researcher compare the "obtained" p-values against?

.05 or .01 to determine whether results are significant or not

What do alpha levels indicate?

how confident you are that the results are not spurious (due to chance or luck) and that you have a solid reliable finding

If obtained p

results are significant

If obtained p > .05...?

results are nonsignificant

What does the obtained p value tell you?

whether mean scores between/among groups are statistically similar or different

whether two VARs are significantly correlated or not

What is significance level not?

a strength of difference

If alpha = .01, what is the probability of getting the same results another time?

99%

The lower the obtained p-value, the lower the probability of making...?

a Type I error

What is a Type I error (false positive)?

rejection of true null Ho

What does a Type I error incorrectly accept?

research Ho

What is a Type II error (false negative)?

rejection of true research Ho

What does a Type II error incorrectly accept?

null Ho

What is different between a .05 and .01 p-value?

the probability of making a Type I error

What are dependent samples t-tests used for?

within group study designs comparing 2 or more unique scores

What are independent samples t-tests used for?

between group study designs comparing 1 or more unique scores

What are 1-way within group ANOVAs used for?

within group study design comparing 3 or more "like" scores (sores related to levels of IV)

What are between group ANOVA, ANCOVA, or MANOVA used for?

between group study design with 3 or more groups comparing 1 or more unique scores

What is the limit on the number of groups being compared on an independent t-test?

2 groups

Is there a limit on the number of variables the groups are being compared on when using an independent t-test?

no

What is the issue with comparing many variables on an independent t-test?

may get spurious (lucky/fake) findings

What must be done to compensate for the chance of spurious findings when more than 2 t-tests are being computed?

set a more stringent alpha level

What does setting a more stringent alpha level do?

ensure any significant difference between groups on any DV is real

What does a more stringent alpha control?

for increased probability of getting significance between at .05 by chance alone

What does adjusting the alpha ensure?

any significance between groups on any given skill is truly significant

What is the alpha adjustment rule?

.05 / # of comparisons (t-tests to run)

What is the omnibus?

the first analysis of an ANOVA

If the omnibus 1-way ANOVA results are significant, what must be done next?

follow-up tukey

What does the follow-up tukey determine?

the Honestly Significant Difference

What does it mean when the variables all show up in different bins on the tukey?

they are all significantly different from each other

What are we always most interested in during mixed-subjects design?

interaction effects

Interaction effects i.e.,

does the active IV affect the group's performance differently?

What does it mean when the active IV affects the group's performance differently?

there is a significant interaction

What needs done when there is a significant interaction during mixed-subjects design?

need to examine the nature of the interaction in follow-up analyses

What question is being asked during the stats of mixed-subjects design?

does the manipulation affect the groups the same or differently

What is the f ratios in mixed-subjects design associated with?

just the groups

What do the f ratios of the between-subjects variable and within-subjects variable pool?

all mean scores

What is mixed-subjects ANOVA always begun with?

omnibus analysis

What does the omnibus analysis of mixed-subjects ANOVA include?

all "a priori" IVs (IVs motivated in research Qs/Hos and design)