applied research skills and stats key terms

p-value

our ability to confidently reject or accept the null hypothesis

questionable research practices

problematic choices whether intentional or not which influence results or conclusions of research

open science movement

centre was established in 2013, aiming to reduce QRPs and make science better and more replicable. training started in 2019

pre-registration

way of reducing QRPs where you specify hypothesis and time stamp them prior to research so you cannot change your mind about direction of research. helps differentiate between confirmatory and exploratory research

correlational research

observing what naturally goes on in the world without interfering. tests association between x and y

third variable problem

association may not be due to the two variables compared. cannot be sure about impact of variables as there’s no manipulation

variance

represents average amount data varies form the mean

spurious correlation

arises when there’s no obvious link between variables but strong correlation is present. may be coincidence or due to QRPs

covariance

measures how two variables vary together e.g. if one rises the other should do the same or opposite depending

correlation coefficient

r. result of a correlational statistical test. it is a standardised statistical index describing the relationship in temps of a direction and size
ratio of how much two variables vary together vs on their own

pearson’s r

used to measure correlation with parametric, continuous data

spearman’s rho

used to measure correlation for non-parametric continuous or ordinal data

kendall’s tau

used for non-parametric correlations with continuous or ordinal data with smaller sample sizes

point biserial

used for parametric correlations with one continuous and one dichotomous variable

non-parametric tests

assumption free tests because they rank the data rather than use the raw scores to remove outliers

replicability

ability for someone else to run same study with new participants and get same results

reproducibility

ability for someone else to re-run your analysis on your participants and get same results

garden of forking paths

there are many reasonable analytic choices that researchers can make with their data which can lead to different results even unintentionally. undermines reproducibility and replicability

analytics flexibility

finding may depend on one specific analytic path that wasn’t reported which would reduce reproducibility and replicability

96%

research found what % of US and Swedish researchers engaged in at least 1 QRP, often presenting same results in different places

one-way within subjects ANOVA

one IV with more than two groups. each participant experiences all levels

one-way between subjects ANOVA

one IV with more than 2 groups where each participant is only in one condition

ANOVAs

aim to understand whether the variability between groups is larger than between them

within group variability

differences between individuals in the group. impacted by individual differences, random noise and measurement errors

between group variability

differences between group means potentially caused by the IV

f-statistic

ratio that measures and compares the amount of variability explained by manipulation and individual differences

total variation

found by calculating difference between each observed data point and the grand mean

total sum of squares

total variance in the data. you square each deviation from mean so that they’re all positive and add together

model sum of squares

how much of the total variation can be explained by the manipulation

residual sum of squares

how much variability cannot be explained by the experimental manipulation

post-hoc test

decided after a significant ANOVA, done on exploratory research to test all combinations

planned comparisons

specified before collection of data for a specific hypothesis

levene’s homogeneity test

done to measure whether the variance in groups is equal to decide on a post-hoc test. if p-value is grater than 0.05 its homogenous

bonferroni test

planned comparison for multiple comparisons and aiming to reduce type 1 error (will reduce power)

LSD

liberal planned comparison test good for when you have a few comparisons and strong predictions, quite high risk of type 1 error

Tukey

for equal group size and unequal or equal group size

games-howell

used for unequal group size and variance

Tukey-kramer

used for equal variance but unequal group size

Mauchly’s test of sphericity

checks if difference between all pairs of conditions is roughly equal
If p is > 0.05 then sphericity is met and no correction is needed
If p is < 0.05 then correction is needed

greenhouse-geisser

method of correction after maulchy’s test of sphericity. done if the epsilon value is less than 0.75. more conservative so is safer

huynh-feldt

done is the epsilon value is greater than 0.75 but can inflate chance of type 1 error

kruksal-wallis

non-parametric one-way between-subjects ANOVA

Friedman’s anova

non-parametric equivalent of one-way within-subjects anova