lecture 3 rmss

jhfbhjfvhbfjvhdfbjfb

sampling strategy

we determine a \____ to collect data form a small subset of the population

small subset of population (sample)

who we use methods and procedures on

inferential level statistics

we use sample data to infer about the population we are interested in

1. limits of our knowledge - we cant know everything about a whole population
2. efficiency - we don't have the time, money, or energy to recruit entire populations

the 2 reasons why we sample?

external validity

in sampling we need broad specific enough data to generalize to population in which we are interested - this contributes to the

internal validity

in sampling we need enough data to learn something about variables this contributes to the

sampling

the singularly most important area of decision-making in social psychology

WEIRD people

whose experiences are represented in social psychology?

WEIRD stands for

western, educated, industrialized, rich, democratic

when Sakaluk critiqued the WEIRD paper he found

the countries they categorized as WEIRD are not actually WEIRDer than the countries they categorized as not

exclusion and non-participation

2 themes of mechanisms that are going on that foster and promote the underrepresentation

exclusion

theme that contributes to underrepresentation - the researcher is not doing outreach in a given community

non-participation

theme that contributes to underrepresentation - researcher might reach out and members of certain communities op to not participate

reasons for underrepresentation: bad structural incentives

they don't care about who you discover a theory with
they just care about what you have discovered
all journals care about is the impact factor so researchers dont make representation a priority

representation and extension of the population

the higher impact factor the journal is the less they care about

reason for underrepresentation: lack of institutional support

journals and universities should call more for generalizability in research

reason for underrepresentation: lack of diversity in faculties

not a lot of representation in university faculties - most members do studies that have something to do with themselves and if universities only cater to one demographic the research only affects the values and concerns of this demographic

reason for underrepresentation: pseudo-methodology

usually include in the limitations reasons why they couldn't get data from an under represented group

reason for underrepresentation: dominant group mediocrity and fragility

over estimate your own utility to a specific community
cant take criticism from the underrepresented group

reason for underrepresentation: history of harms done to community by science

some communities are cynical to the scientific community because they have experiences harm by science and researchers

its deceiving to generalize when using an under-representative sample

one consequence of unrepresentative literature is

Constraints to Generality (COG)

solution to the problem of generalizing findings from an unrepresentative sample to all people?

COGs

we should make people include statement that are clear about who you expect this will apply/generalize to and also be clear about who it wont generalize to

students, internet samples, communities

the 3 places we usually get out sample from in social psychology

good, cheap, fast

the 3 main tradeoffs when choosing a sample - can have 2 but usually not all 3

power

The probability of detecting an effect as "significant" if there is a real effect to be detected

technical definition of power

the probability of not making a Type II/False-Negative error

statistical power

when talking about sample size we are always concerned about

type 1 error

say theres effect when there not - alpha - .05 power is the absence of this

type 2 error

say theres no effect when there is - false negative - beta
.20 is what we strive for
power is absent of this - usually 80% - saying if there is an effect 80% of the time we capture it

anything that increases the value of a test statistic increases power because larger test statistics are more likely to be rejected

what does sample size have to do with power?

bigger n (sample size) \= smaller SE (standard error) \= bigger test statistic \= greater power

how does sample size inform statistical power?

20

Simmons originally said psychologists manufactured significant effects when there is none to be found and we need \___ participants per cell - but in 2018 he came back and had regret because this doesn't give you lots of power to detect reasonable expectations of social psychological effects

250

Schonbrodt and Perugini said we can trust a study if n \= \___

.20

the average social psychological effect is r \=

power calculators calculate

desired power,
desired alpha
desired N
anticipated effect size
whether a one or 2 tailed test will be used

idea behind power calculators

stand alone programs
idea is that there is a known algebraic relationship between type 1 error, type 2 error and anticipated effect size and how you're going to test that effect
know what power you want, what alpha you select
then asks you to estimate the effect size
will you use a one or two titled test
then tell you how many people you will need to have an 80% chance to detect an effect of this size

pros of power calculators

easy to use
friendly to statistcial tests that undergrads preform (correlations and t-tests)

cons of power calculators

You need to a plausible effect size - and this can be hard to get
and they cant handle more complex statistical addtions

power simulations

determining population values for all statistical features of the model you will be testing

how do power simulations work?

You determine how many random samples you want to draw from this population (a really big number, usually \> 10,000) and within each... You fit the model you plan to fit Record what is/isn't significant Rinse and repeat another 9,999 times...
Your power is what % of random samples you detect your (real) effect in

pros of power simulations

Can be designed to match any analytic conditions
Can be designed to give a range of estimates across varied conditions
Also useful for understanding Type I error rates (not just Type II)
if you analyze it you can simulate it

cons of power simulations

you need a good guess for even more than just the anticipated effect size
you need to have a good guess re: all model parameters
requires understanding of programming (e.g., loops) to perform
Sufficiently complex that not all readers will understand

the most scrutinized pieces of method selection

The who, where, and how-many's of sampling

good community based research means

you're either from community, or putting in the work/time to build relationships and learn/center their needs