Research Methods Wolgin Exam 2

Construct Validity

Degree of accuracy in measuring conceptual variables.

Threats to Construct Validity

Factors that compromise the accuracy of constructs.

Threat to Construct Validity: Confounding Variables

Extraneous factors affecting study outcomes.

Threat to Construct Validity: Confounding Variables ~ Reactivity of Subjects

Participants' awareness influencing their behavior.

Threat to Construct Validity: Confounding Variables ~ Random Error of Measurement

Variability from misreading or inaccuracies.

Minimize confounding variables: Operational Definitions

Specific criteria for producing and measuring constructs.

Minimize confounding variables: Protocols

Rules for conducting and observing experiments.

External Validity

Generalizability of study findings to broader contexts (i.e., other situations, populations).

Establishing external validity: Subject Representativeness

Sample's similarity to the larger population.

Establishing external validity: Variable Representativeness

Relevance of chosen variables to the study.

Establishing external validity: Setting Representativeness

Applicability of results to real-world settings.

Internal Validity

Degree of confidence in causal conclusions from a study.

Internal Validity Threat: History

External events (occurring while study is being conducted) affecting study outcomes.

Internal Validity Threat: Maturation

Natural changes in participants over time.

Internal Validity Threat: Testing

Effects of prior measurements on participant's subsequent responses.

Internal Validity Threat: Instrumentation

Changes in measurement tools during study.

Internal Validity Threat: Attrition

Loss of participants affecting study integrity.

Internal Validity Threat: Selection

Pre-existing differences among participant groups.

Confounding

an extraneous factor that systematically varies along with the variables we are studying and therefore provides a potential alternative explanation for our results

How does a confounding variable pose a problem to results interpretation?

Presence prevents us from drawing clear causal conclusion

Reliability

Degree pf consistency of measurement across trials.

Reliability issue: Reliability of Measurements

Every set of measures contains some variability (random error); the more variability, the less reliability

Reliability issue: Statistical Reliability

The likelihood that results are due to chance; if less than 1 in 20 (p < .05) reject possibility of chance

Reliability issue: Experimental Reliability

Replicability of experiments yielding consistent results.

Reliability issue: Test Reliability

Administering the same measures to the same participants on different (two or more) occasions, under equivalent test conditions

Random Assignment

Assign subjects to experimental conditions on a random basis; minimizes confounding

Random Sampling

Equal chance of selection for study participants.

3 criteria by which measurement scales differ: Magnitude of Attribute

Certain scales attribute quantitatively or relatively

3 criteria by which measurement scales differ: Intervals Between Values

Equal vs. unequal/unknwon spacing in measurement scales.

3 criteria by which measurement scales differ: Zero Point

True (i.e., bathroom scale) vs. arbitrary (i.e., temperature) zero in measurement scales.

Nominal Scale

Categorical scale with qualitative differences.

Nominal scale examples

gender, political party

Ordinal Scale

Scale indicating relative differences without equal intervals.

Ordinal scale examples

popularity; ranking

Interval Scale

Scale with equal intervals but no true zero.

Interval scale examples

temperature, calendar date

Ratio Scale

Scale with equal intervals and a true zero point.

Ratio scale examples

money, age, weight

Between Subjects Design

Different groups get different treatments; different participants are assigned to each of the conditions in the experiment.

Within Subjects Design

All subjects get all treatments; each participant engages in every condition of the experiment one or more times.

Controlling a priori differences: Matching Subjects

Matching subjects on some criterion, then random assignment to groups.

Potential problems with matching

May create mismatch on other criteria; subject attrition (loss of subject)

Controlling a priori differences: Randomization

By randomly assigning subjects to groups, confounding variables should be equally distributed.

Carry-Over Effects

When participants' responses in one condition are uniquely influenced by the particular conditions that preceded it.

Order Effects

When participants' responses are affected by the order of the conditions to which they are exposed.

Controlling for carry-over/order effects: Randomization

Randomize the order of treatments

Controlling for carry-over/order effects: Counterbalancing

A procedure in which the order of conditions is varied so that no condition has an overall advantage relative to the other conditions.

Complete Counterbalancing

Each treatment occurs in each time period of the experiment; conditions of an IV are arranged in every possible sequence, equal # of participants assigned to each sequence

Problem with complete counterbalancing

As the number of treatments increases, the number of orders increases disproportionately; therefore, can't run all orders

Incomplete Counterbalancing

Each treatment occurs equally often in each portion of the experiment; an example is a Latin Square design.

Latin Square design

A design where each independent variable shows up only once in each row and column.

How to create balanced Latin Square for within-subjects experiment w/ five levels of IV?

5 columns for each of 5 levels of IV, & 5 rows for each participant; need 2 Latin Squares due to odd # of IVs; must have at least double amt of participants, need min of 10

Latin Square Formula

A, B, C, X, X-1, X-2, etc. where X = final condition

Mixed Design

A factorial design that includes at least one between-subjects variable and at least one within-subjects variable.

Mixed design example

between groups administering a drug via bottle or cannula (one group of participants gets either one or the other) and the within groups would be amphetamine or saline (everyone gets one of each)

IVs in 2 X 4 X 3 between groups design

There are three independent variables.

How many levels of IVs in 2 X 4 X 3 between groups design

There are a total of nine levels of the three independent variables (2 + 4 + 3 = 9)

Main Effect

In a factorial design, when an independent variable has an overall effect on a dependent variable.

Interaction

Occurs when the way in which an independent variable influences a dependent variable differs, depending on the level of another independent variable.

Complete Randomization

Where subjects are randomly assigned to treatment levels or combinations.

Block Randomization

An experimental procedure in which researchers conduct a round of all the conditions, then another round, then another, for as many rounds as needed to complete the experiment; w/in each round, order of condtions is randomly determined

Advantage of Balanced Latin Square over randomization

Each treatment precedes and follows every other treatment equally and every possible confounding effect in the sequence is completely counterbalanced.