Research Methods Review

These flashcards cover key concepts from your lecture notes on identifying variables, distinguishing between different types of claims (frequency, association, causal), understanding various forms of validity, and the criteria for evaluating causal claims.

What is a variable?

A variable is an element, feature, or factor that is liable to change and must have two or more levels (or values).

In the statement 'Most students don’t know when news is false,' identify the variable and its levels.

Variable: knowing when news is false. Levels: knowing and not knowing.

What is a measured variable? Provide an example.

Measured variables are variables whose levels are observed and recorded and cannot be changed (no manipulation). Example: Weight, age, height, IQ.

What is a manipulated variable? Provide an example.

Manipulated variables are variables that a researcher controls by assigning participants to different levels of that variable. Example: Temperature (or amount of sunlight exposure in a plant growth study).

What is a conceptual definition of a variable?

A conceptual definition refers to abstract, theoretical concepts that are 'up in the clouds,' such as infant temperaments or anxiety, which need to be defined to be measured or manipulated.

What is an operational definition of a variable?

An operational definition turns a conceptual definition into a measured or manipulated variable to test a hypothesis with empirical data.

Provide an example of a conceptual variable and its operational definition.

Conceptual variable: spending time socializing. Operational definition: how often a person spends an evening alone, socializes with friends, or sees relatives.

What is a 'claim' in research?

A claim is an argument someone is trying to make.

Define a frequency claim and provide an example.

A frequency claim describes a particular level or degree of a single variable, which is measured, not manipulated. Example: '1/3 of the world smiled today'.

Define an association claim and provide an example.

An association claim states that one level of a variable is likely to be associated with a particular level of another variable, meaning they correlate or covary. These claims help us predict. Example: 'Playing an instrument is linked to cognitive enhancement'.

Define a causal claim and provide an example.

A causal claim states that one variable causes a change in the level of another variable. Example: 'People who eat lunch are not hungry'.

How many variables are typically involved in a frequency claim?

One variable.

How many variables are typically involved in an association claim?

Two variables.

How many variables are typically involved in a causal claim?

Two variables.

Describe a positive association.

A positive association is when the trend points upward, meaning as one variable increases, the other variable also tends to increase.

Describe a negative association.

A negative association is when the trend points downward, meaning as one variable increases, the other variable tends to decrease.

Describe a zero association.

A zero association is when there is no consistent relationship between the variables, resulting in a straight or flat trend.

What does the term 'validity' mean in research?

Validity in research means that the study's conclusions are reasonable, accurate, and justifiable.

List and describe the 'Four Big Validities' in research.

The Four Big Validities are: 1. Construct validity (how well a conceptual variable is operationalized). 2. External validity (how well the results generalize to other people or contexts). 3. Statistical validity (how well the numbers support the claim, including effect size and precision). 4. Internal validity (the extent to which variable A, rather than variable C, is responsible for changes in B).

How do you interrogate the construct validity of a frequency claim?

By asking how well the researcher has measured the variable in question.

How do you interrogate the statistical validity of a frequency claim?

By examining the confidence interval (CI) of the estimate and checking for other estimates of the same percentage.

Is internal validity typically relevant for frequency claims?

No, internal validity is usually not relevant because frequency claims do not assert causality.

How do you interrogate the external validity of a frequency claim?

By asking to what populations, settings, and times the estimate can be generalized, and how representative the sample was (e.g., was it a random sample?).

How do you interrogate the construct validity of an association claim?

By asking how well the researcher has measured each of the two variables in the association.

How do you interrogate the statistical validity of an association claim?

By assessing the estimated effect size (how strong is the association), the precision of the estimate (what is the CI), and what estimates from other studies say.

Is internal validity typically relevant for association claims?

No, internal validity is typically not relevant for association claims as they do not assert causality.

How do you interrogate the external validity of an association claim?

By asking how representative the sample is to generalize the claim, and to what other situations the association can be generalized.

How do you interrogate the construct validity of a causal claim?

By asking how well the researcher has manipulated and/or measured the variables in the study.

How do you interrogate the statistical validity of a causal claim?

By assessing the estimated effect size (how large is the difference between groups) and the precision of the estimate (what is the CI).

How do you interrogate the internal validity of a causal claim?

By examining if the study was an experiment, if it achieved temporal precedence, if it controlled for alternative explanations by randomly assigning participants, and if it avoided internal validity threats.

How do you interrogate the external validity of a causal claim?

By asking if this claim can be generalized, how representative the sample is, and how representative the manipulations and measures are.

Which types of claims do not require internal validity?

Frequency and association claims do not require internal validity.

List and define the three criteria used to evaluate a causal claim.

The three criteria are: 1. Covariance (two variables are observed to go together). 2. Temporal Precedence (the causal variable clearly comes first in time before the effect variable). 3. Internal Validity (the study's method ensures no plausible alternative explanations for the change in the effect variable).

How are covariance, temporal precedence, and internal validity applied to evaluate a causal claim?

1. Two variables must be correlated. 2. The causal variable must come first in time before the outcome. 3. No other explanations must exist for the relationship.

What is a dependent variable?

A dependent variable is the measured variable in a study.

What is an independent variable?

An independent variable is the manipulated variable in a study.

Why might writers' and researchers' claims not always be justified by the studies they are describing?

Claims may not be justified if they pull out correlations which are then assumed to be causal under certain circumstances, or if they prioritize making interesting associations to gather public attention, rather than strictly adhering to what the data truly indicates.

Why is it often difficult for studies to achieve all four kinds of validity at once?

It is difficult because internal and external validities often contradict each other; strengthening one can weaken the other, requiring researchers to prioritize certain validities over others depending on the study's goals.