Experimental Psychology Exam 3

What are parts of a good research hypothesis?

Ideas lead to

• observations

• library research

Statement of problem

Problem statements become research hypotheses when constructs are operationalized

Research Hypotheses

Declarative sentence: Brief and clear

Identifies at least 2 variables

States a predicted relationship with direction

Testing Research Hypotheses

Actually testing three sets of hypotheses

• The null hypothesis

• The confounding variable hypotheses

• The causal hypothesis

Accept causal hypothesis only if you

Reject the null hypothesis (statistical analysis)

Rule out each potential confounding variable hypothesis

(based on appropriate controls)

Types of Validity

Statistical Validity

Construct Validity

External Validity

Internal Validity

Statistical Validity

Are the statistical tests accurate?

Threatened by

• Unreliable measures

• Violations of statistical assumptions

Strengthened by

• Using well-validated measures

• Not violating assumptions of the statistic (e.g., linearity, normal distribution, equal group sizes, etc.)

Construct Validity

Threatened by

• Any alternative explanation for the results

Strengthened by

• Using well-validated constructs to build the theoretical predictions for the study

Internal Validity

Is the independent variable responsible for the observed changes in the dependent variable?

Threatened by

• Confounding variables

Strengthened by

• Adding adequate controls to reduce or eliminate

confounding

External Validity

Do the results apply to the broader population?

Threatened by

• Unrepresentative samples

• Generalizing beyond the limits of the sample

Strengthened by

• Gathering a representative sample (if possible)

• Clearly describing the sample, so that other researchers will know the limits of generalization

internal validity vs external validity

INTERNAL VALIDITY - The degree to which a researcher controls for and reduces the effects of extraneous variables that can affect study outcomes so that they represent true outcomes.

EXTERNAL VALIDITY - The degree to which results from an experiment can be generalized to other individuals beyond the study.

What are the 9 threats to internal validity?

1. History

2. Maturation

3. Testing

4. Instrumentation

5. Regression of the mean

6. Selection

7. Attrition

8. Diffusion of treatment

9. Sequence Effects

History

Changes due to an event that occurs during the study

Maturation

Changes due to growth or predictable changes

Testing

Changes due to the effects of previous testing

Instrumentation

Any change in the calibration of the measuring instrument over the course of the study

Regression of the mean

Tendency for participants selected because of extreme scores to be less extreme on a retest

Selection

Any factor that creates groups that are not equal at the start of the study

Attrition

Loss of participants during a study; are the participants who drop out different from those who continue?

Diffusion of treatment

Changes in participants' behavior due to information they obtained about other conditions

Sequence Effects

Effects on performance in one condition due to experience with previous conditions

Subject Effects

Participants are not passive

• _______: Changes in behavior due to being in a study and not the IV

• Demand characteristics

Placebo effect

Demand characteristics:

Cues given to participants on how the researcher expects them to behave

Placebo effect

Treatment effect due to expectations that the treatment will work

Experimenter Effect

Any biasing effects in a study that are due to the actions of the researcher

Based on the expectations of the researcher

It can affect the outcome of studies if not controlled

Maybe due to the experimenter providing demand characteristics to the participant

Not the same as scientific fraud (which is deliberate)

Pretest-Posttest Design

Single-group, pretest-posttest design compares pre-treatment and post-treatment scores to determine whether improvement occurs.

The simple pretest-posttest design fails to control many sources of confounding

• History, Maturation, Regression to the Mean, etc.

However, it fails to control most sources of confounding.

Two-group, Posttest-only Design

Compares different treatments on an outcome measure

Much stronger than a single-group, pretest-posttest design

Control Procedures

General control procedures (applicable to virtually all research)

Control over subject and experimenter effects

Control through the selection and assignment of participants

Control through experimental design

General Control Procedures

Preparation of the setting

Response Measurement

Replication

Preparation of the setting

Free of distractions that might interfere

A natural setting increases external validity

Response Measurement

Use reliable and valid measures

Replication

Demonstrates that findings are consistent and robust

Subject and Experimenter Effects

Blind procedures

Automation

Using objective measures

Multiple observers

Using deception

Balanced Placebo Design

Blind procedures

Best control for expectancy effects

Single-blind

Double-blind

Single-blind

The experimenter does not know what condition the participant is in

Double-blind

Neither the experimenter nor the participant knows what condition the participant is in

Automation

Reduces contact between participants and the experimenter

Gives the experimenter less opportunity to affect participants

Using objective measures

Objective measures require less judgment

Provides less opportunity for subtle experimenter biases to affect the data

Multiple observers

Reduces bias because it challenges observers to be as precise and objective as possible

Can measure the amount of observer agreement (percent agreement or Kappa)

Using deception

Hides the purpose of the study from participants

Balanced placebo design is a good example

Balanced Placebo Design

Separates the pharmacological effects from the expectancy effects of alcohol

A two-factor design

• Factor 1 is whether the person drinks alcohol

• Factor 2 is whether the person thinks he or she is drinking alcohol

Participant Selection

Can generalize only if your sample is representative

Populations and samples

• General population

• Target population

• Accessible population

• Sample

General population

all potential participants

Target population

those participants you are interested in

Accessible population

portion of target population that is available to the researcher

Sample

drawn from the accessible population

Random sampling

Every participant has an equal chance of being sampled

Stratified random sampling

Random sampling within strata (subgroups)

Ad hoc samples

Random sample from accessible population

Must generalize cautiously

Should describe a sample to help define the limits of generalization

Convenience sampling

choosing individuals who are easiest to reach

Free random assignment

Random assignment of participants to groups

Randomize within blocks

Randomly assign in blocks of one participant per condition

Matched random assignment

Random assignment of participants in matched sets to groups

Matched Random Assignment

Match on relevant variables

• Variables likely to affect the dependent measure

• Variables that show the largest variability in the population Procedures

• Match in sets on the relevant variable

• Set size is the number of groups in the study

• Randomly assign participants from the set, one to each group• Keep track of matching data for the statistical analysis

Experimental Design

Experimental design maximizes validity

• Need to also include the other control procedures covered in this chapter

Key elements of experiments

• One or more control groups

• Random assignment of participants to groups

Randomized Pretest-posttest Control-group Design

The control-group design controls most sources of confounding, PROVIDED the groups are equivalent

• Use random assignment to assure equivalence