crim 220 - lecture 3

Measuring Constructs and Operationalization

Construct Representation: When conducting research, abstract concepts (constructs) need to be measured using specific variables.
- Example: General academic performance can be represented by variables like course rating, overall GPA, or cumulative GPA over the last two years of university.
Recidivism Measurement: Measuring recidivism, which refers to individuals being re-arrested or re-offending after release from prison, is not straightforward.
- Considerations:
  - Frequency: A single instance of re-arrest might not be sufficient to understand the extent of recidivism. Researchers often need to observe more than one instance to build a robust inquiry.
  - Time Span: The time frame for observing recidivism is crucial.
    - A shorter study period (e.g., six months post-program) might make a program appear more effective because fewer instances of re-offending are likely to occur.
    - A longer study period (e.g., five years) offers a more realistic view, potentially showing more offenses, but also provides a more valid assessment of long-term program effects.

Empirical Questions: Causal vs. Descriptive

Implicit Assumptions: Research questions often involve implicit assumptions about cause and effect, where one thing is thought to cause another.
Two Main Types of Empirical Questions:
1. Descriptive Questions: Aim to simply describe a situation for a population without establishing cause and effect.
  - Examples:
    - "What is the HIV prevalence rate in Canada?"
    - "What is the number of people who are going to cheat next year?"
    - "How would raising taxes by 100\% affect the motivation to cheat on taxes?" (This describes a potential outcome, but doesn't necessarily establish a direct, isolated cause.)
2. Causal Questions: Focus on establishing that one variable causes a change in another.

Criteria for Causation

To confidently state that something causes something else, three criteria must be satisfied:

Relationship/Correlation: There must be a demonstrable relationship between the two variables of interest; they must be correlated.
- They must "vary together," meaning that as one variable increases or decreases, the other variable changes in a predictable manner.
- Example: The amount of studying is related to grades. As studying increases, grades tend to increase.
Temporal Precedence: The cause must occur before the effect.
- It must be clear which variable came first. Without this, it's difficult to establish a causal direction.
- Challenge: Sometimes, it's hard to tell whether A caused B or B caused A, or if both are effects of a third factor.
Non-Spuriousness: The observed correlation between the two variables must not be due to a "sneaky third variable" that was not measured.

Spurious Relationships

A spurious relationship occurs when two variables appear to have a causal connection, but in reality, they do not. Instead, an unmeasured third variable is related to both the perceived cause and effect, creating the illusion of a direct causal link.

Validity: General Definition

Validity refers to whether statements about a given cause and effect are true (valid) or false (invalid).
When something is deemed valid, it means there is sufficient relevant evidence to support the inference that a claim or conclusion is true or correct. It asks: "Do we actually have evidence to support that claim?"

Types of Validity

1. Conclusion Validity

Definition: Our ability to determine whether a change in the cause (independent variable) is statistically correlated with a change in the effect (dependent variable).
Example: Comparing exam grades between a group that studied for 2 hours and another that studied for 20 hours. The hypothesis is that the 20-hour group would have higher grades.
- Sample Size Impact: If the sample size is too small (e.g., 5 students per group), the test may lack the ability to find a true difference. A larger sample (e.g., 50 students per group) is more likely to yield a valid answer.

Statistical Power

Definition: The probability that a statistical test will be able to find a statistically significant difference between groups in your sample, if there is, in fact, a statistically significant difference between the groups in the population.
- This is a crucial concept related to sample size.
Importance: In most studies, researchers examine a sample, not the entire population. There's always a chance that sample findings might not fully represent the population.
Example: Studying drinking and driving rates among male versus female high school students across British Columbia (BC).
- If you only sample one school in Surrey and one in Burnaby, you hope these findings generalize to wider BC high school students.
- The power of your statistical test would be the probability it finds a statistically significant difference between males and females at those two high schools, given that such a difference exists in the entire province of BC.
Relationship with Sample Size: Generally, the bigger the sample taken from a population, the more statistical power you will have.
Target Power: Academically, a power of at least 0.80 (80\%) is conventionally desired. This means an 80\% chance of detecting a true effect if one exists.
Power Analysis:
- A Priori (Prospective) Power Analysis: Conducted before the study to determine the necessary sample size to achieve a desired power (e.g., 0.80) for a given effect size and significance level.
- Post Hoc (Retrospective) Power Analysis: Conducted after the study to determine the power achieved given the actual sample size and observed effect size. If a study found no significant difference, it might be due to a lack of power rather than an actual absence of difference.

2. Internal Validity

Definition: The extent to which conclusions can be drawn about the causal effects of one variable on another. An observed association between two variables has internal validity if the relationship is genuinely causal and not due to the influence of one or more "sneaky extra variables."
Consequences of Low Internal Validity: Leads to biased study findings.
Sources of Internal Validity Threats: Often arise from the effects of one or more other variables systematically impacting the relationship being studied. These are sometimes called "systematic influences."
- Example: If students with a higher IQ are inherently more likely to perform well on an exam regardless of study hours, IQ becomes a systematic influence. If IQ isn't accounted for, a study might wrongly conclude that studying hours are the primary cause of exam performance, or overestimate their impact.

3. External Validity

Definition: The extent to which the observed results can be generalized to other populations, settings, times, and alternative ways of measuring variables.
Example: A study on cell phone distractions while driving.
- Experiment: Four experiments conducted by David Strayer in 2003, using 110 undergraduate students with hands-free phones in a driving simulator, found drivers on phones were more accident-prone and slower to react.
- Concern for External Validity: The sample of 110 undergraduate students (typically aged 18-24) may not be representative of other populations.
  - Question: Do these results apply to drivers over the age of 65? Older individuals may react physically slower or process information differently, which could affect the generalizability of findings from a younger sample.

4. Construct Validity

Definition: Concerned with how well the observed relationship between the variables a researcher has measured represents the underlying causal process of interest.
This refers to the ability to generalize from what we are observing and measuring (e.g., specific test scores) to the more abstract theoretical constructs (e.g., intelligence or academic ability) the study aims to investigate.
Importance: Construct validity, along with external validity, are key concepts frequently discussed in academic papers and require clear understanding.