test 2

Bivariate Correlational Designs

• Definition: Involves two variables that are measured, without manipulated variables.

• Types of Associations:

  • Direction:

  • Positive: Increases in one variable relate to increases in another.

  • Negative: Increases in one variable relate to decreases in another.

  • Magnitude:

  • Strong: High correlation values (near to 1 or -1).

  • Weak: Low correlation values (near to 0).

Identifying Correlational Designs

• Association Claims: Avoid causal language.

• Language Used for Association Claims:

  • “____ is linked to ____”

  • “____ is related to ____”

  • “____ is associated with _____”

  • “People who _____ have _____”

  • “People who _____ are _______”

Identifying Variables

Meaningful Conversations Linked to Happier People

• Predictor Variable(s): Meaningful conversations

• Outcome Variable(s): Happiness

Couples Who Meet Online Have Better Marriages

• Predictor Variable(s): How couples met (online vs. offline)

• Outcome Variable(s): Marital satisfaction

Evaluating Association Claims

Criteria for Evaluation

1. Construct Validity

2. External Validity

3. Statistical Validity

4. Internal Validity

Construct Validity

• Definition: Are predictors and outcomes measured well?

• Example: Measurement for the study about selfish people; items related to otherish motivation were measured with statements rated on a four-point scale from strongly disagree to strongly agree.

• Measures:

  • Otherish motivation scale (Cronbach's alpha = .58)

  • Number of children: Measured as “How many children have you ever had?”

  • Income: Measured as annual family income before taxes; log transformed.

Sample Evaluating Association Claims

• Mehl et al., 2010

• Method: 79 undergraduates wore the EAR device, recording conversations.

• Reliability: Interrater reliability for conversation type coded resulted in ICC = .84.

External Validity

Definition

• External Validity: Do the results generalize to other samples?

Study Example

• Study on selfishness pulled from the General Social Survey (GSS) with a representative sample of 5,294 individuals.

Moderator Variables

• Definition: A variable that changes the strength or direction of the relationship between two other variables.

• Example: Differences in the relationship between deep talks and well-being depending on whether talks took place during the weekday or the weekend.

Statistical Approach

• Importance of matching how variables are measured with statistical tests conducted.

• Types of Variables to Assess:

  • Continuous (ratio, interval)

  • Categorical (nominal, ordinal)

Associations

• Continuous vs. Continuous Example: A correlation between wellbeing and percentage of deep talk; quantified by scatterplots showing data spread.

• Categorical vs. Continuous Example: A t-test comparing social time for depressed vs. non-depressed individuals represented by bar graphs.

Statistical Validity

Key Questions

1. How strong is the relationship between the two variables?

2. How precise is the estimate?

3. Has the study been replicated?

4. Are there factors impacting the correlation?

Factors That Impact Validity

• Outliers: Extreme scores influencing correlation results.

• Sample Size: Size affects the precision of correlation estimates.

• Curvilinearity: Non-linear relationships not accurately captured by simple correlation metrics.

• Restriction of Range: Lack of full variable value range impacting correlation.

• Reliability: Unreliable measures reflect erroneous correlations.

• Method Variance: Inflation of correlation due to shared measurement methods.

Statistical SignificanceDefinition

• An association is statistically significant when:

  • p < .05;

  • The 95% confidence interval does not contain zero.

  • Example: Relationship between how people met and marital satisfaction r = .06, 95% CI [.05, .07], p < .05 is statistically significant but indicates a small effect.

Power

• Definition: Power refers to the ability to detect an effect if one exists.

  • Larger sample sizes provide enhanced power and reduce Type II errors.

Internal Validity

Definition

• A correlational study does not possess internal validity, preventing causal claims from being made.

• Key Points:

1. Covariance: Must show correlation

2. Temporal Precedence: Cause must precede effect

3. Internal Validity: Must rule out alternative explanations

Directionality Problem

• The uncertainty of which variable occurred first complicates causal claims.

• Example: Does selfishness lead to fewer children or is it the other way around?

Third Variable Problem

• The presence of alternative explanations for variable relationships (confounds) impacts causal interpretations.

Criteria for Establishing Causality

1. Covariance: The results must show a correlation between the two variables.

2. Temporal Precedence: The causal variable must precede the effect variable in time.

3. Internal Validity: The study design must rule out plausible alternative explanations for the relationship between the two variables.

Reasons Casual Claims Cannot Be Made from Correlational Studies

• Directionality Problem: Uncertainty about which variable came first (i.e., simultaneous measurement of both variables).

• Third Variable Problem: There may be an alternative explanation for the relationship between the two variables, such as a confounding variable.

Significant Thoughts on Establishing Causality

• To establish causality, all three criteria (covariance, temporal precedence, internal validity) must be satisfied.

• Only a well-designed experimental study can meet all three criteria for establishing causality.

Correlational Designs

• Bivariate Correlational Design: Focuses on the relationship between two continuous variables.

  • Analyses like t-tests can be used when involving one categorical and one continuous variable.

  • Example: Measuring the relationship between time spent watching violent shows and aggression levels.

Directionality Problem in Bivariate Correlational Designs

• Measurement takes place at the same time, leaving the sequence of variables ambiguous.

• Questions arise, such as: Did watching violent TV shows come first, or did aggression come first?

Longitudinal Designs

• Longitudinal designs are multivariate correlational studies where the same variables are measured over time to help establish temporal precedence.

Key Correlational Statistics and Interpretations

• Cross-sectional Correlation & Autocorrelation:

  • Cross-sectional correlation: Examines correlations at the same time.

  • Autocorrelation: Looks at stability over time.

• Correlations may be statistically significant if p < .05.

Cross-lagged Correlation

• A method used in longitudinal designs to investigate whether earlier measures of one variable correlate with later measures of another variable.

Implications of Longitudinal Designs

• Autocorrelation: Measures the same variable’s stability over time.

• Cross-lagged correlation: Assesses the relationship over time in two different variables, controlling for time.

  • Question Addressed: Can earlier measures of one variable predict later measures of another?

Ruling Out Third Variables

1. Statistical Control: Holding constructs constant to measure unique effects post-analysis.

2. Experimental Control: Keeping variables constant across experimental conditions.

   • Multiple regression is a common method used to statistically control third variables.

Multiple Regression

• This statistical analysis evaluates the impact of one or more predictors while controlling for others.

• Results indicate the effect of covariates on outcomes while measuring significant relationships.

Interpretation of Multiple Regression Results

• A particular link remains significant even when controlling for additional variables (e.g., empathy).

• If a link disappears after considering a third variable, it indicates that the third variable might explain the relationship.

Moderation

• Examines if the relationship strength between two variables changes across different groups or contextual factors.

• Example: The effectiveness of study methods on exam performance might vary based on the type of studying (practice retrieval vs. reviewing notes).

Mediation

• Investigates underlying mechanisms between two correlated variables, explaining why they are related.

• Steps of Mediation:

  1. Confirm that the predictor correlates with the outcome.

  2. Confirm the predictor correlates with the mediator.

  3. Confirm the mediator correlates with the outcome (controlling for the predictor).

  4. Evidence that the predictor has a diminished relationship with the outcome when controlling for the mediator.

Distinct Paths in a Mediation Model

• Path a: Link between predictor (e.g., lack of sleep) and mediator (e.g., perspective-taking ability).

• Path b: Link between mediator and outcome (e.g., relationship conflict).

• Path c: Total effect of the predictor on the outcome.

• Path ab: The indirect effect of the predictor on the outcome through the mediator.

• Path c': Direct effect of the predictor on the outcome when controlling for the mediator.

Proportions of Effect in Mediation

• Determining proportions adds depth to understanding relationships.

  • Complete mediation: Proportion mediated ≥ 0.80.

  • Partial mediation: Proportion mediated < 0.80.

Establishing Covariance

• Comparison Group:

  • Necessary for answering the question “compared to what?”

  • Example data: Treatment vs. Control groups measuring materialism (scale).

Establishing Temporal Precedence

• Ensure IV manipulation occurs before measuring DV.

Establishing Internal Validity

• Confound:

  • Variable that varies systematically with the levels of the IV and could affect the DV.

  • Example: Participants’ mood influencing results.

• Noise:

  • Unsystematic variance that does not correlate systematically with IV levels, affecting DV unpredictably.

Internal Validity Issues

• Selection Effect:

  • Occurs when types of participants differ systematically between groups (especially problematic if participants choose conditions).

• Random Assignment:

  • Helps mitigate confounds by transforming potential confounds into noise variables.

• Matched Groups Design:

  • Participants are matched on key characteristics before random assignment to conditions, eliminating selection effects.

• Independent-groups designs (between-groups):

• Post-test only.

• Pre-test & post-test.

• Matched-groups design.

• Within-groups designs:

• Repeated measures.

• Concurrent measures.

Combatting Internal Validity Threats

• Use comparison groups actively.

• For threats such as maturation and history, check participant baseline characteristics.

• Ensuring uniformity in DV measurement is crucial across timepoints.

Confounds:

• Variables that may influence results alongside the IV.

• Selection Effects:

• Differences in participant type between groups.

• Order Effects:

• Repeated measures can introduce fatigue, practice, or carryover effects. Mitigated by counterbalancing.

• Researcher Bias:

• Expectations of researchers can skew results; addressed by double-blind studies.

• Demand Characteristics:

• Participants modify behaviour due to perceived study purposes; mitigated by double-blind designs.

• Placebo Effects:

• Changes noted in participants solely based on belief they are treated; managed with placebo controls.

• Maturation Threat:

• Changes occurring naturally over time.

• History Threat

• External events affecting participants simultaneously.

• Regression Threat:

• Extreme scores revert to average levels upon repeat measurement.

• Attrition Threat:

• Loss of participants that skews results when dropouts are non-random.

Association Claims

• Association Claims: Describe a relationship between variables without suggesting causation. Examples include:

  • “Linked,” as in the case of meaningful conversations and happiness.

  • “Are,” as seen with online dating and marriage quality.

• Key Characteristics of Association Claims:

  • Do not imply that one variable causes another.

  • Use verbs that suggest relationships, not causes.

• Defining Multivariate Correlational Research:

• Involves examining relationships among more than two variables to identify patterns or associations without manipulation.

Case Study: Parental Praise and Narcissism

• Narcissism Defined: A personality trait characterized by egotism, need for admiration, and lack of empathy.

• Overpraise: When parents excessively compliment children, suggesting superiority over peers.

• Research Findings:

  1. Covariance confirmed by Otway & Vignoles (2006) study.

  2. No temporal precedence in their methodology as both variables were measured simultaneously.

  3. Third-variable explanations were possible, such as parental characteristics impacting praise.

Multiple Regression Analysis

• Purpose: Helps in ruling out third variables affecting relationships.

• Key Terms:

  • Criterion Variable: Dependent variable researchers are focusing on (e.g., pregnancy risk).

  • Predictor Variables: Independent variables considered during analysis (e.g., sexual content, age).

  • Controlling for Variables: Holding a potential third variable constant while examining other associations.

Criteria for Establishing Causation Through Longitudinal Designs

• Covariance: Longitudinal designs can show variable relationships with confidence intervals that exclude zero.

• Temporal precedence: Clear differentiation of timing when measuring events.

• Internal validity: Use designs that clarify the roles of third variables, further supported by separate analyses if required.

The Really Bad Experiment (Cautionary Tale)

• Three Fictional Experiments:

  • Nikhil's Study: 15 boys showed behavioral changes after diet changes, but alternative interpretations (natural behavior maturation) are possible.

  • Dr. Yuki's Study: 40 women with depression showed lower symptoms post-therapy, yet spontaneous remission may explain results.

  • Go Green Campaign: University dorm reduced energy usage; seasonal adjustment could be an alternative explanation.

• Common Design Template: All examples show lack of proper controls (i.e., comparison groups) leading to unreliable conclusions on causation.

Maturation Effects Explained

• Description: Maturation refers to improvements in participants' behavior that occur naturally as they adjust to their environment.

• Examples:

  • Nikhil's campers could have improved behavior due to adaptation rather than diet changes.

  • Depression symptoms in Dr. Yuki's study might have improved over time without treatment.

• Prevention Strategy: Including a control group would help rule out maturation effects by comparing with similarly aged participants not receiving the treatment

History Effects Explained

• Description: History threats arise from external events that influence the group’s behavior during the experiment.

• Example: Go Green Campaign results may appear due to seasonal temperature drops rather than campaign effectiveness.

• Prevention Strategy: Measuring a comparable group’s outcomes during the same period to account for external factors.

Regression to the Mean Explained

• Concept: Extreme values on repeated measures are likely to move closer to the average on subsequent measures.

• Example: Women's depression scores in Dr. Yuki's study might show improvement due to their initial extreme levels rather than the treatment effect.

• Prevention Strategy: Include an appropriate comparison group, ensuring groups are equally extreme at pretest.

Attrition Effects Explained

• Concept: Participant dropout can skew results, particularly if dropouts have extreme scores.

• Example: If the most unruly campers drop out of Nikhil's study, results could falsely appear as though the intervention was effective.

• Prevention Strategy: Adjust posttest averages based on participants who completed the study, analyzing dropout characteristics.

Testing Effects Explained

• Concept: Familiarity with a test can improve results or fatigue may degrade posttest scores.

• Example: Performance may improve on a second attempt not due to treatment but due to repetition.

• Prevention Strategy: Use alternative measures across different testing sessions or omit pretests in favor of posttests only.

Instrumentation Effects Explained

• Concept: Changes in measurement tools or procedures can alter findings.

• Example: Coders may become more lenient over time, creating misleading data analysis.

• Prevention Strategy: Ensuring all measuring tools are calibrated and retraining personnel as needed.

Observer Bias and Demand Characteristic

• Observer Bias: When researchers’ interpretations are skewed by their expectations, affecting internal validity.

• Demand Characteristics: Participants change behavior due to awareness of the study purpose, thus confounding results.

• Solutions: Implement double-blind or masked designs to shield both participants and experimenters from biases.

Null Effects and Their Implications

• Understanding Null Effects: Reflects scenarios where no statistically significant difference is found.

• Investigating Null Causes: Potential obscuring factors relate to manipulations or measurements being ineffective, variability within groups being too high, or no true effects existing.

• Importance of Reporting: Null results must be transparently documented to inform scientific understanding and guide future research.

Addressing Obscuring Factors

• Two Main Strategies:

  1. Increase between-group distinctions by improving manipulations and clarifying dependent measures.

  2. Reduce within-group variability through larger sample sizes, repeated measurements, and controlled experimental environments.