research methods unit 2

chapters 7-12

chapter 6

surveys and observations

survey

a method of posing questions to people on the telephone, in personal interviews, on written questionnaires, or via the Internet

poll

a method of posing questions to people on the telephone, in personal interviews, on written questionnaires, or via the Internet

forced-choice questions

a survey question format in which respondents give their opinion by picking the best of two or more options

likert scale

a survey question format using a rating scale containing multiple response options anchored by the specific terms strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree. a scale that does not follow this format exactly is called a i Likert-type scale .

semantic differential format

a survey question format using a response scale whose numbers are anchored with contrasting adjectives.

open-ended questions

a survey question format that allows respondents to answer any way they like

heading questions

a type of question in a survey or poll that is problematic because its wording encourages one response more than others, thereby weakening its construct validity

double-barreled question

a type of question in a survey or poll that is problematic because it asks two questions in one, thereby weakening its construct validity

negatively worded questions

a question in a survey or poll that contains negatively phrased statements, making its wording complicated or confusing and potentially weakening its construct validity

response sets

a shortcut respondents may use to answer items in a long survey, rather than responding to the content of each item

acquiescence

answering yes or strongly agree to every item in a survey or interview (also known as yea-saying)

fence-sitting

playing it safe by answering in the middle of the scale for every question in a survey or interview

observational research

the process of watching people or animals and systematically recording how they behave or what they are doing.

observer bias

a bias that occurs when observer expectations influence the interpretation of participant behaviors or the outcome of the study

observer effects

a change in behavior of study participants in the direction of observer expectations. (also called expectancy effect)

masked design

A study design in which the observers are unaware of the experimental conditions to which participants have been assigned. (also called blind design)

a method of asking people questions about their attitudes, beliefs, or behaviors using questionnaires or interviews.

survey research

advantages of surveys?

efficient, inexpensive, can measure thoughts/attitudes directly, large samples

disadvantages of surveys?

people may lie, misunderstand questions, or show response bias

watching behavior in a real-world setting without interfering

naturalistic observation

researcher becomes part of the group being studied

participant observation

structured observation

watching behavior in a controlled or structured situation

when to use participant observation?

when studying group behavior or culture from the inside

the degree to which two or more observers agree

interrater reliability

guidelines explaining how to identify and record behaviors

coding manuals

clearly defining a variable so it can be measured

operationalization

chapter 7

sampling: estimating the frequency of behaviors and beliefs

population

a larger group from which a sample is drawn; the group to which a study’s conclusions are intended to be applied

sample

the group of people, animals, or cases used in a study; a subset of the population of interest

census

a set of observations that contains all members of the population of interest

biased sample

A sample in which some members of the population of interest are systematically left out and, therefore, the results cannot generalize to the population of interest

unbiased sample

a sample in which all members of the population of interest are equally likely to be included (usually through some random method), and therefore the results can generalize to the population of interest

convenience sampling

choosing a sample based on those who are easiest to access and readily available; a biased sampling technique

self selection

a form of sampling bias that occurs when a sample contains only people who volunteer to participate

probability sampling

a category name for random sampling techniques, such as simple random sampling, stratified random sampling, and cluster sampling, in which a sample is drawn from a population of interest so each member has an equal and known chance of being included in the sample

nonprobability

a category name for nonrandom sampling techniques, such as convenience, purposive, and quota sampling, that result in a biased sample

chapter 11

more on experiments: confounding and obscuring variables

one group pretest/posttest design

an experiment in which a researcher recruits one group of participants; measures them on a pretest; exposes them to a treatment, intervention, or change; and then measures them on a posttest.

maturation threat

a threat to internal validity that occurs when an observed change in an experimental group could have emerged more or less spontaneously over time.

history threat

a threat to internal validity that occurs when it is unclear whether a change in the treatment group is caused by the treatment itself or by an external or historical factor that affects most members of the group.

regression threat

a threat to internal validity related to regression to the mean, a phenomenon in which any extreme finding is likely to be closer to its own typical, or mean, level the next time it is measured (with or without the experimental treatment or intervention).

regression to the mean

a phenomenon in which an extreme finding is likely to be closer to its own typical, or mean, level the next time it is measured, because the same combination of chance factors that made the finding extreme are not present the second time.

attrition threat

in a pretest/posttest, repeated-measures, or quasi-experimental study, a threat to internal validity that occurs when a systematic type of participant drops out of the study before it ends.

testing threat

in a repeated-measures experiment or quasi-experiment, a kind of order effect in which scores change over time just because participants have taken the test more than once; includes practice effects.

instrumentation threat

a threat to internal validity that occurs when a measuring instrument changes over time.

selection-history threat

a threat to internal validity in which a historical or seasonal event systematically affects only the participants in the treatment group or only those in the comparison group, not both.

selection-attrition threats

a threat to internal validity in which participants are likely to drop out of either the treatment group or the comparison group, not both.

observer bias

a bias that occurs when observer expectations influence the interpretation of participant behaviors or the outcome of the study.

demand characteristics

a cue that leads participants to guess a study’s hypothesis or goal, a threat to internal validity.

double-blind study

a study in which neither the participants nor the researchers who evaluate them know who is in the treatment group and who is in the comparison group

masked design

a study in which observers are unaware of the experimental conditions to which participants have been assigned

placebo effect

a response or effect that occurs when people receiving an experimental treatment experience a change only because they believe they are receiving a valid treatmen

null effect

a finding that an independent variable did not make a difference in the dependent variable; there is no significant covariance between the two.

ceiling effect

an experimental design problem in which independent variable groups score almost the same on a dependent variable, such that all scores fall at the high end of their possible distribution.

floor effect

an experimental design problem in which independent variable groups score almost the same on a dependent variable, such that all scores fall at the low end of their possible distribution.

manipulation check

in an experiment, an extra dependent variable researchers can include to determine how well a manipulation worked.

noise

unsystematic variability among the members of a group in an experiment, which might be caused by situation noise, individual differences, or measurement error. also called error variance, unsystematic variance.

measurement error

the degree to which the recorded measure for a participant on some variable differs from the true value of the variable for that participant. Measurement errors may be random, such that scores that are too high and too low cancel each other out; or they may be systematic, such that most scores are biased either too high or too low.

power

the likelihood that a study will show a statistically significant result when an independent variable truly has an effect in the population.

what are three possible threats to internal validity?

design confounds, selection effects, and order effects

design confound

there is an alternative explanation because the experiment was poorly designed

another variable happened to vary systematically with the intended independent variable

design confound

selection effect

a confound exists because the different independent variable groups have systematically different types of participants

order effect

there is an alternative explanation because the outcome might be caused by the order in which the levels were presented

what is the actual name of the really bad experiment?

one-group pretests/posttest design

what are six potential internal validity threats (in one-group pretest/posttests)?

maturation effects, history effects, regression threats, attrition effects, testing threats, and instrumentation threats

what are two obscuring factors that might occur when there is a null result in an experiment?

not enough between-groups difference or too much within-groups variability

what are two changes that can be made in a within-groups experiment to correct for individual differences?

change the design or add more participants

chapter 12

experiments with more than one variable

interaction effect

a result from a factorial design, in which the difference in the levels of one independent variable changes, depending on the level of the other independent variable; a difference in differences.

factorial design

a study in which there are two or more independent variables, or factors.

cells

a condition in an experiment; in a simple experiment, a cell can represent the level of one independent variable; in a factorial design, a cell represents one of the possible combinations of two independent variables.

participant variable

a variable such as age, gender, or ethnicity whose levels are selected (i.e., measured), not manipulated.

main effect

in a factorial design, the overall effect of one independent variable on the dependent variable, averaging over the levels of the other independent variable.

marginal means

in a factorial design, the arithmetic means for each level of an independent variable, averaging over the levels of another independent variable.

participant variable

a variable such as age, gender, or ethnicity whose levels are selected (i.e., measured), not manipulated.

what do experiments with multiple independent variables show?

interactions

when we say we are looking for a difference in differences what are we asking/testing?

we are asking if the the effect of one IV depends on the level of another IV

“it depends”

crossover interaction

“especially when..”

spreading interaction

crossover interaction

a type of statistical interaction where two independent variables have opposite effects on a dependent variable, resulting in an "X-shaped" graph where lines cross

spreading interaction

a statistical interaction in factorial designs where the effect of one independent variable exists at one level of a second independent variable, but is significantly weaker or nonexistent at the other level

what type of interaction is this?

crossover

what type of interaction is this?

spreading

two or more independent variables (or factors)

factorial design

moderator

a variable that changes the relationship between 2 other variables

what type of factorial design is this?

2 × 2 (two by two)

factorial designs test the generalizability of a causal variable AND

theories about how variables interact with one another

factorial designs test theories about how variables interact with one another AND

the generalizability of a causal variable

which is more important when interpreting results, a main effect or an interaction?

the interaction

an independent factorial design is also known as

a between subjects factorial

a within groups factorial design is also known as

a repeated measures factorial

independent groups factorial design

both independent variables are studied aqs independent groups

within groups factorial design

both independent variables are manipulated as within groups

mixed factorial design

one independent variable is manipulated as independent groups and another is manipulated as within groups

what type of factorial design is this?

2 × 3 (two by three)

what type of factorial design is this?

3 × 4 (three by four)

what type of factorial design is this?

three way design or 2 × 2 × 2 (two by two by two)

in a three way design how many differences are we looking for?

three main effects (one for each independent variable), plus three separate two-way interactions and a three-way interaction