SCMA Chapter 6 Data Collection Strategies

A researcher wants to study the effect of a new tutoring program. They introduce the program in one Grade 10 class and compare it to another existing Grade 10 class, without random assignment. What type of research design is this?

C) Quasi-experiment

In a true experiment, what is the primary purpose of a control group?

B) To provide a comparison for determining whether change may be due to the treatment.

A researcher records students' weekly paid-work hours and their GPAs to determine if the two variables are related, without manipulating either variable. What type of research is this?

C) Correlational

Random assignment in an experiment guarantees that the experimental and control groups are identical in every way before the treatment is administered.

False

Quasi-experimental designs are considered to have higher internal validity than true experiments.

False

Experiment

A method of data collection designed to test hypotheses under controlled conditions, with the goal of eliminating threats to internal validity.

True Experiment

A design with three key features: independent and dependent variables, pretesting and post-testing, and experimental and control groups.

Independent Variable (IV)

The variable that the researcher manipulates.

Dependent Variable (DV)

The outcome that is measured to see if it changes.

Pre-Test

Measurement of the dependent variable before the experimental manipulation.

Post-Test

Measurement of the dependent variable after the experimental manipulation.

Experimental Group

The group that receives the treatment or manipulation.

Control Group

The group that does not receive the treatment; treated identically otherwise; its purpose is to control for rival plausible explanations.

Random Assignation

Every participant has an equal chance of being assigned to each condition, independently of others.

Pre-Test Equivalence

The assumption that the experimental and control groups are equal in all respects before the independent variable is administered.

Block Randomization

All conditions occur once in the sequence before any of them is repeated; conditions occur in random order within each block.

Field Experiment

An experiment conducted in real-life surroundings; stronger external validity, weaker control.

Laboratory Experiment

An experiment in an artificial environment; greater control, easier random assignment, easier replication, weak external validity.

Internal Validity

The extent to which the design supports the conclusion that changes to the IV were responsible for changes in the DV.

External Validity

Whether findings are applicable outside the research environment and can be generalized beyond the people studied.

Threats to Internal Validity

History, Testing, Instrumentation, Mortality, Maturation, and Selection.

History Threat

An event occurring after the treatment that may influence the dependent variable.

Testing Threat

The pre-test itself may influence the dependent variable.

Instrumentation Threat

Changes in the way a test is administered may account for pre-test and post-test differences.

Mortality Threat

Participants leaving the experiment before it is over.

Maturation Threat

Participants changing over time (e.g., getting older, developing).

Selection Threat

Post-test differences between groups may be caused by pre-existing differences.

Threats to External Validity

Selection x Treatment, Setting x Treatment, History x Treatment, Pretesting Effects, Reactive Effects of Experimental Arrangements.

Reactive Effects

Subjects behaving atypically because they know they are in an experiment.

Non-Experimental Research

Research that lacks manipulation of an independent variable and/or random assignment.

Cross-Sectional Research

Compares two or more pre-existing groups; the IV is not manipulated and there is no random assignment.

Correlational Research

The researcher is interested in the relationship between variables but does not attempt to influence them.

Pearson's r

A correlation coefficient measuring the strength of linear relationships; ranges from -1.0 to +1.0.

Observational Research

Seeks to explore an aspect of the world by watching behavior.

Covert Observation

Participants do not know they are being observed; avoids changed behavior but carries ethical problems.

Overt Observation

Participants know they are being observed and give their approval.

Quasi-Experiment

An experiment-like design where the IV is manipulated but participants are not randomly assigned; selection is by participants, researcher, or both.

Natural Experiment

Experiment-like conditions produced by naturally occurring phenomena or changes made by people not doing research.

Three Criteria for Causality

The IV comes before the DV in time; X and Y are related; the relationship is not explained by other causal agents.

Selection Bias

Pre-existing differences between comparison groups that may account for results.

Regression Discontinuity Design

Technique for reducing selection bias in a quasi-experiment.

Propensity Score Matching

Technique for reducing selection bias in a quasi-experiment.

Longitudinal Design

Cases examined at one time (T1) and again at others (T2, T3); provides information about the time-order of changes.

Panel Study

The same people, households, or organizations studied at different times.

Cohort Study

People sharing an experience studied at different times, but different people may be studied each time.

Attrition

Participants dropping out of a longitudinal study over time.

Panel Conditioning

People's attitudes and behaviors change because of participating in a panel.

Case Study Design

An in-depth study of a single case: a person, family, organization, event, or country.

Internal Validity Ranking

Experimental (highest) > Quasi-experimental (middle) > Correlational (lowest).

Nomothetic Explanation

An explanation of cause and effect expressed as general laws; typically quantitative.

Idiographic Explanation

A rich description of a particular person or group; typically qualitative.