Methods of Empirical Research Exam 2

Experiment

Experiment

a study in which an intervention is deliberately introduced to observe its effects

counterfactual

what the experimental group participants' responses would have been if they had not received the treatment

selection bias

personal choices by subjects that create differences between them

causal effect requirements

The cause preceded the effect • The cause was related to the effect • We can find no plausible alternative explanations for the effect other than the cause

confound

third variable that causes both the hypothesized independent variable and the dependent variable

Randomized experiment

an experiment in which units are assigned to receive treatment or an alternative control by a random process

Quasi experiment

an experiment in which units are not assigned to conditions randomly

Natural experiment

Not really an experiment because the cause cannot be manipulated; a study that contrasts a naturally occurring event such as an earthquake with a comparison condition

Correlational study

usually synonymous with nonexperimental or observational study; a study that simply observes the size and direction of a relationship among variables

independent variable

variable that is manipulated; sometimes called the predictor variable

dependent variable

The outcome factor; the variable that may change in response to manipulations of the independent variable

experimental conditions

varying levels or values of treatment applied to the independent variable

control group

the group that does not receive the experimental treatment.

experimental group

the group in an experiment that receives the variable being tested

subject (participant) variable

a characteristic inherent in the subjects that cannot be changed

simple random assignment

placing participants in experimental conditions in such a way that every participant has an equal chance of being placed in any condition

matched random assignment

participants are matched into homogenous blocks, and then participants within each block are assigned randomly to conditions

within-subjects design

participants are exposed to all levels of the independent variable

order effects

A confounding variable arising from the order which participants take place in the different conditions e.g. boredom, sensitization, fatigue

carryover effects

some form of contamination carries over from one condition to the next

systematic variance

the portion of the total variance in a set of scores that is related in an orderly, predictable fashion to the variables the researcher is investigating

error variance

The amount of variability among the scores caused by chance or uncontrolled variables.

internal validity

the degree to which changes in the dependent variable are due to the manipulation of the independent variable

external validity

extent to which we can generalize findings to real-world settings

attrition

a natural loss of individuals within a study

history effect

Occurs when some change other than the experimental treatment occurs during the course of an experiment that affects the dependent variable.

experimenter's dilemma

the trade-off between internal and external validity

one-way design

an experimental design with a single independent variable

factorial design

a study in which there are two or more independent variables, or factors

Design specification

An indication of the number of independent variables, each independent variable's number of levels, and the total number of conditions

main effect

In a factorial design, the overall effect of one independent variable on the dependent variable, averaging over the levels of the other independent variable.

interaction

caused by differing effects of one independent variable across the levels of other independent variables (ex: the effects of a sleeping pill on various amounts of alcohol consumption)

expericorr factorial design

an experimental design that includes one or more manipulated independent variables and one or more preexisting participant variables that are measured rather than manipulated; also called mixed factorial design

moderator variable

a factor that alters the strength or direction of the relation between an independent and dependent variable

mediating variable

a variable that helps explain the relationship between two other variables; caused by the key independent variable

Limits to experimental research in social sciences

Manipulable factors • Ethical concerns • Resources and time • Participant buy-in

Molar causation

An interest in the overall causal relationship between a treatment package and its effects, in which both may consist of multiple parts

Molecular Causation

an interest in knowing which parts of a treatment package are more or less responsible for which parts of the effects through which mediational processes

crossed design

each level of each independent variable is crossed with all levels of all other independent variables

nested design

some levels of one factor are not exposed to all levels of the other factors; can lead to influence between groups

significance testing

Testing to determine the probability of there being a "true" difference between scores of a data set. Involves testing a null hypothesis in order to prove it, and therefore disprove the difference.

null hypothesis

a statement or idea that can be falsified, or proved wrong; states that the independent variable did not have an effect on the dependent variable; opposite of the experimental hypothesis

p-value

The probability level which forms basis for deciding if results are statistically significant (not due to chance).

alpha level

The threshold P-value that determines when we reject a null hypothesis

Type I error (alpha)

False positive results; ex: reject the null hypothesis when you should accept it

Type II error

An error that occurs when a researcher concludes that the independent variable had no effect on the dependent variable, when in truth it did; a "false negative"

Power

the likelihood that statistically significant systematic variance will be found; related to the number of participants used

effect size

the magnitude of a relationship between two or more variables

Pearson correlation

measures the degree and the direction of the linear relationship between two variables

Cohen's d

a measure of effect size that assesses the difference between two means in terms of standard deviation, not standard error

Odds Ratio (OR)

the ratio of an event occurring in one group to the odds of it occurring in another group; requires at least two levels of dependent variable

confidence interval

a plausible range of values for the population mean

Point estimation

most likely value of the population mean from the sample data

margin of error

distance from the mean to one end of the confidence interval

regression to the mean

the tendency of extreme scores on a variable to be followed by, or associated with, less extreme scores

maturation bias

changes due to passage of time, not intervention

John Henry Effect

control group alters behavior due to knowledge that they're in the control group; confound

Hawthorne effect

the alteration of behavior by the subjects of a study due to their awareness of being observed; confound

comparative study

Specific type of nonexperimental study that includes a categorical variable (ex: gender differences in math performance)

Scatterplot

a graphical depiction of the relationship between two variables; each unit corresponds to a point on the graph

Pearson correlation (r) properties

varies from -1 to 1, with sign indicating direction of relationship and number representing magnitude

directional hypothesis

States the direction of the difference or relationship

nondirectional hypothesis

Predicts the existence of a relationship, not its direction

on-line outlier

outlier on both variables, inflates strength of correlation

Off-line outliers

fall outside of the pattern of the rest of the data and tend to artificially deflate r

spurious correlation

an apparent but false relationship between two (or more) variables that is caused by some other variable

partial correlation

the correlation between two variables with the influence of a third variable statistically controlled for

regression analysis

extending the correlation between variables to make a prediction of one variable on another; accuracy is dependent on strength of relationship

criterion variable

dependent variable in regression analysis

predictor variable

independent variable

regression equation

A formula for a line of best fit that models a linear relationship between two quantitative variables; y=a+bx

multiple regression

a statistical technique that includes two or more predictor variables in a prediction equation

standard multiple regression

all of the predictor variables are entered into the regression analysis at the same time

stepwise multiple regression

a multiple regression analysis in which predictors enter the regression equation in order of their ability to predict unique variance in the outcome variable

hierarchal multiple regression

predictors are entered into the regression equation in a predetermined order based on the hypothesis

fit index

indicates how well the hypothesized model fits the data

multilevel modeling

statistical techniques that can describe relations between variables when data are varied on multiple levels

factor analysis

correlations among many variables are analyzed to identify closely related clusters of variables

latent variable

a variable that is not directly observed but is inferred or estimated from observed variables; underlying factor

factor matrix

A table of correlations between variables and factors; the correlations are called factor loadings.

crud factor

everything correlates to some extent with everything else

evidence of causal theory

Malleability • Temporal ordering • Control variables