Research Methods Exam 2

validity of measurement

an assessment of how well we measured what we wanted to measure

validity of a research study

an assessment of the quality of the research process and the accuracy of the results

internal validity

extent to which study provides evidence of a cause effect relationship between IV and DV

construct validity

extent to which results supportthe theory (and only that theory) behind the research

external validity

extent to which results generalizes to other situations and people

ecological validity

degree to which an experimental situation resembles the real world

statistical validity

extent to which data reveals true rather than accidental (chance) effects

extraneous variable (EV)

any vaiable in a study other than the two variables studied (eye color, foods liked)

confounding variable (CV)

extraneous variable changes systematically with the 2 variables being studied

what are the 11 threats to internal validity?

1. history

2. maturation

3. effects of repeat testing

4. regression to the mean

5. selection bias

6. mortality

7. diffusion/imitation of treatment

8. compensatory equalization

9. compensatory rivalry

10. resentful demoarlization

11. experimental bias

what is history pertaining to internal validity?

when environmental effects other than your treatment occur between administration of your treatment; affecting at least one treatment differently.

what is maturation pertaining to internal validity?

any physiological or psychological changes in participants that are unrelated to the IV; problematic for long designs using young children/elderly

effects of repeative testing?

when being tested influences performance on subsequent testing

practice effect?

if performance is better after Treatment 2 is that because Treatment 2 promotes better performance or had more practice with the task?

fatigue effect

if perfomance is better from Treatment 1 is that because Treatment 1 promotes better performance or because they were tired/unmotivated during Treatment 2?

what is regression to the mean?

tendency for subject w/ extreme (high/low) scores to score closer (regress) to the mean on subsequent testing

what is selection bias?

when bias in selecting groups leads to non-equivalant groups; participants in one group differ from another (age, IQ, motivation)

why is selection bias a problem?

can’t tell if differences between the groups on the DV are due to the IV or due to differences among the participants

what is mortality?

differential attrition/selective subject loss; when participants from Group A drop-out more than those from Group B; if the high IQ group drops out creates lower for the rest

what is diffusion-imitation of treatment?

when treatment spreads from Group A to Group B; through subjects with big mouths; you learned PEMDAS from others

what is compensatory equalization?

group A demands (and receives) treatment that Group B receives; heard of benefits and demands it

what is compensatory rivalry?

Group A, which doesn’t get treatment, tries to outperform Group B; motivation level becomes the confounding variable

waht is resentful demoralization?

opposite of another one: Group A, which doesn’t get treatment, resents Group B and underperforms; not gonna do well so doesn’t try

what is participant expectancy?

when participants don’t behave “naturally” because they expect certain results; due to their expectations and not your theoretical explanation

what is demand characteristics?

cues to the purpose of your study that influence participants behavior

what is good subject tendency?

act in accord with what they think you want; pick up on nature

what is social desirability effect?

behave in a socially desirable fashion

what is the threat other subjects relating to external validity?

will result from your subjects generalization to other subjects; college students »» married middle-aged spouses?

what is the threat other times relating to external validity?

will results from Era A generalize to Era B?

what is the threat other settings relating to external validity?

will result found in lab generalize to the “real world”, outside of that context?

what is 1 threat to statistical validity?

Low Power - the likelihood that a statistical test will allow researchers to correctly reject the null hypothesis; if IV has an effect then high power study is more likely than a low power study to find that effect; if study has low power than cna conclude your results are inconclusive

experimental group

subjects who receive a certain level of the IV

comparison group (s)

similar subjects who receive a different level of the IV; or nothing at all

What are the logic of a between-subjects design?

groups alike, manipulate IV, and resulting differences between groups must be due to IV

what is control?

any means used to rule out threats to a study’s validity

what is between-subjects design?

one group of subjects gets one level of the IV while the other group(s) get the other levels of IV; comparison done between groups of subjects

what are the 5 ways to establish equivalant groups?

1. hold variables constant across groups; same room, experimenter, day

2. restrict range of variability; only subs with IQs of 100-110

3. randomly assign subjects to groups

4. match subjects between groups

5. use a within-subjects research designs

what is randomly assigning subjects to groups?

all subjects have an equal chance of being assigned to each gp

what is matching a subject between groups?

say you want to ensure that your two groups are equivalent on a variable; measure each participant on that variable, list participants form highest to lowest score and randomly assign the members of each pair to the groups

what is using a within-subjects design?

all Ss receive all levels of IV presented at different times; comparison done within subjects and serve as your own comparison

what are two advantages of within-subjects design?

1. requires fewer Ss (subjects); huge when participants are hard to obtain

2. largely eliminates individual different problems; ultimate in equivalent groups

what are the four disadvantages of within-subject design?

1. participant attrition - can result in only one measurement for a subject

2. time-related problems - participants may be influenced by outside factors between measurements; lowers internal validity

3. order effect problems (contrast effect problems) - first measurement may influence subsequent measurements; lowers internal validity

4. cant be used w/ the manipulations that are lasting; training wheels vs downhill

what is experimental research?

• subjects are randomly assigned to groups

IV is __ during experimental research?

manipulated

Effect on DV is ____ during experimental research?

measured

what is nonexperimental research?

research in which the experimenter does NOT have control over the conditions of the study

what are the 4 types of experimental research?

1. observational

2. archival

3. case study

4. survey/interview

what is observational experimental research?

IV is not manipulated

what is archival experimental research?

existing data

what is case study experimental research?

single person

what is survey/interview experimental research?

chap 9

what is observational research?

observe and systematically record behavior; observation & description only, tells us what can happens NOT why it happens, no manipulation; causation NOT determined

what are the 2 types of observational research?

naturalistic observation and participant-observer observation

what is naturalistic observation?

careful monitoring and examination of what animals do in “real world” conditions; Jane Goodall

what is participant-observer observation?

researcher engages in same activites as people being observed; invasion of privacy, act of observation can change the situation and behavior

what is archival research?

research using existing data that the researcher had no part in collecting

what is case study?

thorough observation and description of single individual; observation and description only, tells us what can happen, NOT what typically happens and NOT why ; causation NOT determined

What is a survey?

a method of gathering info. by asking questions to some people

how to construct a survey - use already established one?

should be reliable and valid

how to construct a survey - create one?

harder than it seems

how to construct a survey - think carefully about types of q’s and responses you want?

open-ended, close-ended (yes/no), partially open-ended (yes/no/other for explain), rating scaled

how to construct a survey - questions should?

be clearly worded, use simple, appropriate language

how to construct a survey - avoid using?

leading questions, loaded emotion-laiden questions (abortion), double barreled questions (2 questions in 1), a series of questions worded in a similar manner

how to construct a survey - things to consider?

how will you administer it? - individually, groups, mail, online, verbal surveys can be face-to-face, on phone or in groups; respinse rate is very important

what is social desirability?

when people respond to surveys as they think they “should” rather than what they truthfully feel or believe

what is monotonicity of response?

people sometimes just circle the same response over and over and over - insufficiently attending to the survey; these issues compromise the validity of your data

What are 5 ways to increase the veracity of responses?

1. include a social desirable scale - assess the degree to which respondents endorse social-desirable responses

2. ensure an anymity - not able to tell = more honest

3. repeat questions and look for deviation in respones

4. look for monotincity of responses ahead of time in your data set

5. measure the amount of time it takes to complete the survey

what is a population?

draw inferences about; entiere set of every potential subject who falls into our category of interest, who we want our findings to generalize to

what is a sample?

collect data from; a group of subjects (drawn from the relevant pop) who participate in our study

what is a biased sample

if not a representative of the population; age, IQ, sex

probability sampling

each member of the population has a specific probability of being chosen for the sample - must know popsize, sampling done via random selection

non-probability sampling

dont know the probability of any one person being chosen for the sample - sampling done via convenience, ease, has greater risk of producing biased samples

what is simple random sampling?

every member of a population has an equal chance of being selected as a member of the sample - clearly define pop, list all members of pop, and using random process to select member from the list of sample

what is stratified random sampling?

sample takes into account different types of people in the population - select subgroups from pop to be included in sample, then select an equal random ample from each sub group then combine in one sample

what is proportronate stratified random sampling?

proportions of different types of people in sample are identical to proportions in pop - select subgroups from pop to be included in sample, determine what proportion of pop corresponds to each subgroup, then select proportionate random sample from each subgrouping, then combine into one sample

what is convenience sampling?

subjects selected based on their accessibility and our convenience