Exam 3 Vocabulary Research Methods

survey

a method of posing questions to people online, in-person, or in written questionnaires

forced-choice question

a survey question format in which respondents give their opinion by picking the best of 2 or more options

likert scale

a survey question format using a rating scale containing multiple response options anchored by specific terms

i-likert scale

likert scale that does not follow the exact criteria for the specific terms a likert scale uses

open-ended question

a survey question format that allows respondents to answer anything

leading question

a type of question in a survey that is problematic because its wording encourages one response more than others, weakening its construct validity

double-barreled question

a type of question in a survey or poll that is problematic because it asks two questions in one, weakening its construct validity

negatively worded questions

a question in a survey or poll that contains negatively phrased statements, making its wording complicated or confusing and potentially weakening construct validity

response set

a shortcut responders may use to answer items in a long survey, rather than thinking carefully about each question, weakening construct validity because the participant is not saying what they really think

acquiescence

answer yes or strongly agree to every item, weakening construct validity because participant is not saying what they really think

fence sitting

playing it safe by answering in the middle of the scale on all items, weakening construct validity because participant is not saying what they really think

socially desirable responding

giving answers on a survey that make you look better than you really are

observational research

the process of watching people or animals and systematically recording how they behave or what they are doing

observer bias

a bias that occurs when an observer’s expectations influence their interpretation of participant behaviors or study outcomes

observer effect

a change in behavior of study participants in the direction of what they believe the observer’s expectations are

codebooks

clear instructions to help researchers and observers make reliable judgments with low bias; precise statements of how the variables are operationalized

masked design

a study design in which observers are unaware of the experimental conditions to which observers have been assigned

reactivity

a change in the behavior of study participants because they are aware they are being watched

unobtrusive observation

an observation in a study made indirectly through physical traces of behavior, or by someone hidden/posing as a bystander

participant observation

a qualitative research technique in which one or more researchers live among the population they are studying

informant

in qualitative research, a person who is expert in the area of interest and who works with the researcher to understand a social issue or other question

in-depth interview

qualitative method where researchers prepare a set of topics they want to learn about, and meet with a sample of informants for interview sessions that are recorded and transcribed

focus group

interview conducted in groups of 6-10 people with experience relevant to a research question

population

a larger group from which a sample is drawn; the group to which a study’s conclusions are intended to be applied

sample

the group of people, animals, or cases used in a study; a subset of the population of interest

census

a set of observations that contains all members of the population of interest

representative sample

sample where all members of population are equally likely to be included, so results can generalize to the population

unrepresentative sample

sample where some members of the population are systematically left out, so results cannot generalize to the population

convenience sampling

choosing a sample based on those who are easiest to access and readily available

self-selection

a form of sampling bias that occurs when a sample contains only people who volunteer to participate

probability sampling

a category name for random sampling techniques, in which a sample is drawn from a population of interest so each member has an equal and known chance of being included in the sample

simple random sampling

the most basic form of probability sampling, in which the sample is chosen completely at random from the population of interest

systematic sampling

a probability sampling technique in which the researcher uses a randomly chosen number N, and counts off every N^th member of the population to achieve a sample

cluster sampling

a probability sampling technique in which clusters of participants within the population of interest are selected at random, followed by data collection from all individuals in each cluster

multistage sampling

a probability sampling technique involving at least 2 stages- a random sample of clusters followed by a random sample of people within the selected clusters

stratified random sampling

a form of probability sampling; a random sampling technique in which the researcher identifies particular demographic categories and then randomly selects individuals within each category

oversampling

a form of probability sampling; a variation of stratified random sampling in which the researcher intentionally overrepresents one or more groups; occurs when the proportion in sample would not be enough to make statistical estimates

weighting

if researchers determine that the final sample contains fewer members of a subgroup than it should, they adjust data so responses from members of underrepresented subgroups count more and overrepresented groups count less

purposive sampling

a biased sampling technique in which only certain kinds of people are included in a sample

snowball sampling

a biased sampling technique in which participants are asked to recommend acquaintances for the study; a variation on purposive sampling

quota sampling

a biased sampling technique in which a researcher identifies subsets of the population of interest, sets a target number for each category in the sample, and nonrandomly selects individuals within each category until quotas are filled

reliability

the consistency of the results of a measure

validity

the appropriateness of a conclusion or decision

test-retest reliability

the consistency in results every time a measure is used

interrater reliability

the degree to which 2+ coders/observers give consistent ratings of a set of targets

internal reliability

in a measure that contains several items, the consistency of a pattern of answers no matter how a question is phrased

correlation coefficient r

a single number, ranging from -1.0 to 1.0, that indicates the strength and direction of an association between 2 variables

strength

a description of an association indicating how closely the data points in a scatterplot cluster along a line of best fit drawn through them

average inter-item correlation

a measure of internal reliability for a set of items; mean of all possible correlations computed between each item and the others

cronbach’s alpha

a correlation-based statistic that measures a scale’s internal reliability

face validity

the extent to which a measure is subjectively considered a plausible operationalization of the conceptual variable in question

content validity

the extent to which a measure captures all parts of a defined construct

criterion validity

whether the measure under consideration is associated with a concrete behavioral outcome that it should be associated with

known-groups paradigm

when researchers see whether scores on the measure differ between two groups that are known to differ on the variable of interest, to ensure they differ using this measure too

convergent validity

an empirical test of the extent to which a self-report measure correlates with other measures of a theoretically similar construct

discriminant validity

an empirical test of the extent to which a self-report measure does not correlate strongly with measures of theoretically dissimilar constructs

bivariate correlation

an association that involves exactly two variables

effect size

magnitude and strength of relationship between two or more variables

restriction of range

in a bivariate correlation, the absence of a full range of possible scores on one of the variables, so the relationship from the sample underestimates the true correlation

curvilinear association

as one variable increases, the level of the other first increases, then decreases

directionality problem

in a correlational study, the occurrence of both variables being measured around the same time, making it unclear which variable in the association came first

third-variable problem

in a correlational study, the existence of a plausible alternative explanation for the association between two variables

spurious association

a bivariate association that is attributable only to systematic mean differences on subgroups within the sample; the original association is not present within the subgroups

moderator

a variable that, depending on its level, changes the relationship between two other variables

multivariate design

a study designed to test an association involving 2+ measured variables

longitudinal design

a study in which the same variables are measured in the same people at different points in time

cross-selectional correlation

in a longitudinal design, a correlation between 2 variables that are measured at the same time

autocorrelations

in a longitudinal design, the correlation of one variable with itself, measured at two different times

cross-lag correlations

in a longitudinal design, a correlation between an earlier measure of one variable and a later measure of another variable

multiple regression

a statistical technique that computes the relationship between a predictor variable and a criterion variable

control for

holding a potential third variable at a constant level while investigating the association between two other variables

criterion variable

the variable in a multiple regression analysis that the researchers are most interested in understanding or predicting

predictor variable

a variable in a multiple regression analysis that is used to explain variance in the criterion variable

parsimony

the degree to which a theory provides the simplest explanation of a phenomenon; simplest explanation of a pattern of data; best explanation that requires the fewest exceptions or qualifications

pattern and parsimony

a pattern of results best explained by a single, parsimonious causal theory

one-group, pretest/posttest design

experiment in which a researcher recruits a group of participants, measures them on a pretest, exposes them to a treatment, and then measures them on a posttest

maturation threat

a threat to internal validity that occurs when an observed change in an experimental group could have emerged spontaneously over time

history threat

a threat to internal validity that occurs when it is unclear whether a change in the treatment group is caused by the treatment itself or by an external/historical factor that impacts most of the group in the same direction

regression threat

a threat to internal validity related to regression to the mean, where any extreme finding is likely to be closer to the mean when measured next; only occurs when a group is measured twice and has extreme scores at pretest

attrition threat

a threat to internal validity when a specific type of participant systematically drops out of the study before it ends

testing threat

a kind of order effect in which scores change over time just because participants have taken the test more than once

instrumentation threat

a threat to internal validity that occurs when a measuring instrument changes over time or different forms of the test are used for pre/post, but are not equal

selection-history threat

a threat to internal validity in which a historical/seasonal event systematically impacts only one group, not both

selection-attrition threat

a threat to internal validity in which participants are more likely to drop out of one group, not both

quasi-experiment

research design used to estimate causal relationships without random assignment; participants are not randomly assigned to conditions, but an independent variable is still manipulated

waitlist design

an experimental design for studying a therapeutic treatment, in which researchers randomly assign some participants to receive the therapy immediately, and others receive it after a time delay

inferential statistics

a set of techniques that uses the laws of chance and probability to help researchers make decisions about what their data means and what inferences they can make from the data

estimation

an approach to inferential statistics that uses data from a sample to calculate an effect size and a 95% confidence interval, with the goal of predicting the magnitude of some value in a population

null hypothesis significance testing

an inferential statistics technique in which a result is compared to a hypothetical population in which there is no relationship or no difference

point estimate

a single estimate of an unknown population parameter based on sample data

confidence interval

a range of values that often contains the true population level; for 95% of infinite possible samples of the same size, the interval around the sample value will capture the true population value

standard error

the typical/average error researchers make when estimating a population value; measures the amount of discrepancy that can be expected in a sample estimate vs. the true value in the population

sampling distribution of the mean

a hypothetical distribution you would get if you conducted the same study an infinite number of times and plotted the estimates you got

p value

in null-hypothesis significance testing, the probability of getting the result in a sample or one more extreme, by chance, if there is no relationship or difference in the population

null hypothesis

assume there is no effect in the population (starting point of statistics)