Notes on Research Methods: Descriptive, Correlational, and Experimental

Observer Bias and Research Design

  • Research objective: understand how manipulating independent variables affects outcomes like work performance (e.g., speed and quality).

  • Independent variable: the variable that the researcher changes/manipulates.

  • Dependent variable: the outcome that is measured.

  • Operational definition: precise, objective description of how variables are measured.

  • Example: studying whether observation affects productivity; observe whether being watched increases output.

  • Observer bias:

    • A systematic error in observation due to the observer's expectations.
    • Can lead to reporting a behavior differently than it actually occurred.
    • Goal: keep observers objective and reduce bias through design and procedures.

  • Correlational studies (vs. experimental):

    • Purpose: identify relationships between variables.
    • Limitation: Cannot infer causation. Correlation does not imply causation.
    • Common caution: media claims about a medication curing or causing something may be based on correlational data; such claims lack causal evidence without experimental support.
    • Ethical constraint: some causal questions (e.g., what causes PTSD from trauma) cannot be studied via random assignment to a harmful condition.
    • Practical note: strong correlations can guide practice and policy (predictions and risk assessment) even without causation.

  • Interpreting correlations:

    • When one variable changes, the other tends to change in a related way, but not necessarily because one causes the other.
    • Direction of relationship:
    • Positive correlation: both variables move in the same direction (increase together or decrease together).
    • Negative correlation: variables move in opposite directions.
    • Zero correlation: no linear relationship.
    • Visual aid: scatter plots graphically represent two-variable relationships.
    • Common misconception: stronger correlation (closer to ±1) indicates stronger evidence of causality, which is not true.

  • Why not always use experimental designs?

    • Some experiments would require unethical conditions (e.g., exposing people to trauma) and cannot be randomly assigned.
    • When random assignment is not possible, researchers rely on correlational or quasi-experimental designs.
    • Example ethical constraint: studying PTSD and combat trauma requires correlational approaches because you cannot randomize people to experience trauma.

  • Prediction and risk assessment (applied correlational work):

    • Strong associations can help predict outcomes and inform decisions (e.g., suicide risk assessment from depressive symptoms).
    • Prediction relies on the strength of relationships, not on proving causation.
    • Real-time, clinical decisions may depend on identifying risk factors and protective factors through correlational data.

  • Experimental design basics (to establish causation):

    • Random assignment: every participant has an equal chance of being placed in any condition.
    • Control vs. experimental group:
    • Experimental group: receives the treatment/independent variable manipulation.
    • Control group: receives no treatment or a placebo/neutral intervention.
    • Confounds: any uncontrolled factor that could influence the dependent variable and threaten internal validity by offering alternative explanations.
    • Goal: ensure the only systematic difference between groups is the independent variable.
    • Example caveat: a study testing wall color on test performance must randomize room assignment to avoid confounds like time of day or self-selection bias.
    • Random assignment vs random sampling:
    • Random assignment: equal chance of being in any condition (improves internal validity and generalizability).
    • Random sampling: ensures the sample represents the population for external validity.

  • Population vs. sample:

    • Population: everyone the researcher is interested in.
    • Sample: a subset of the population actually studied.
    • Example: population = all individuals diagnosed with autism; sample = the 100 diagnosed individuals in the study.

  • Ethics and human/animal research:

    • IRB (Ethics Review Board): approves research proposals to protect participants from unreasonable harm.
    • Informed consent: participants are informed about what will happen in the study and consent to participate; minors require parental/guardian consent.
    • Deception and debriefing: deception may be used but must be followed by debriefing; participants are told the true purpose after the study when appropriate.
    • Risk-benefit analysis: weighs potential risks against the importance of the knowledge gained.
    • Anonymity vs. confidentiality:
    • Anonymity: researchers do not collect identifying information, data cannot be traced back to individuals.
    • Confidentiality: data are stored securely and kept private; identities are protected but may be possible to identify under certain safeguards.
    • Participant rights: to withdraw at any time without penalty; researchers cannot coerce participation or revoke credits for leaving early.
    • SONA system and outside activity credits: opportunities for psychology students to participate in research to fulfill course/unit requirements.

  • Research validity and reliability (core quality metrics):

    • Validity: accuracy of what a study claims to measure.
    • Construct validity: extent to which the variables measure what they are intended to measure.
    • External validity: generalizability of findings to other settings, populations, or times.
    • Internal validity: degree to which observed effects are due to the manipulation of the independent variable, not confounds.
    • Reliability: consistency of measurement across time and items.
    • If a measure is reliable but not valid, it is consistently wrong; if valid, it must also be reliable.
    • Descriptive statistics and data quality:
    • Mean: ar{x} = rac{1}{n}
      \sum{i=1}^n xi
    • Median: middle value when data are ordered; for even n, average the two middle values.
    • Mode: most frequently occurring value.
    • Range: extRange=extmax(x)−extmin(x)ext{Range} = ext{max}(x) - ext{min}(x)
    • Standard deviation (sample): s =
      \sqrt{\frac{1}{n-1}\sum{i=1}^n (xi - \bar{x})^2}
    • Correlation coefficient ( Pearson r):
      r=∑(x<em>i−xˉ)(y</em>i−yˉ)∑(x<em>i−xˉ)2  ∑(y</em>i−yˉ)2r = \frac{\sum (x<em>i - \bar{x})(y</em>i - \bar{y})}{\sqrt{\sum (x<em>i - \bar{x})^2}\; \sqrt{\sum (y</em>i - \bar{y})^2}}

  • Supplemental concepts and tools:

    • Meta-analysis: study of many studies on the same topic; combines results to provide stronger evidence than a single study.
    • Scatter plots: graphical display of the relationship between two variables; used to assess strength and direction of correlations.
    • Interpreting correlation strength by context; stronger correlations approach the ±1 boundary, but direction alone does not imply causation.
    • Common-sense cues and metaphors:
    • A cartoon example: correlation does not imply causation (e.g., more time in a room with a broken screen does not prove who caused it).

  • Practical takeaways for evaluating information:

    • Look for evidence of random assignment, control groups, and pre-registered analysis to support causal claims.
    • Be wary of extraordinary claims without rigorous experimental support or clear limitations.
    • Always check for stated limitations and whether the authors discuss potential confounds.

  • How to read research reports like a practitioner:

    • Identify the independent and dependent variables and how they are operationalized.
    • Check for threats to internal validity (confounds) and how they were controlled.
    • Note the type of validity claimed and whether the conclusions match the design.
    • Examine the sample and population to assess external validity.
    • Review ethical considerations: consent, confidentiality, risk/benefit, and deception/debriefing.

  • Final practical example connections to practice:

    • In clinical practice, correlational data can inform risk assessment (e.g., depression levels and suicide risk) but cannot establish that depression causes suicide on its own.
    • When designing interventions, randomized experiments provide stronger evidence for causal effects but may be constrained by ethics; hence, researchers often rely on well-conducted correlational studies and natural experiments.

  • Things to remember (quick recap):

    • Observers should avoid bias; use objective measures and blinding where possible.
    • Correlation does not equal causation; strong relationships guide but do not prove causal links.
    • Experiments with random assignment and proper controls can establish causality.
    • Ethics (IRB, consent, deception/debriefing, confidentiality) are central to responsible research.
    • Validity and reliability are the backbone of trustworthy data; understand their nuances and how they affect conclusions.

  • Simple recall prompts for study sessions:

    • Define independent, dependent, and operational definitions.
    • Explain why correlational studies cannot prove causation.
    • Describe a confound and give an example.
    • Distinguish between internal and external validity with examples.
    • List and explain the three measures of central tendency and when to prefer each.
    • State the formula for the correlation coefficient and interpret the meaning of r = ±0.75.
    • What is a meta-analysis and when is it more convincing than a single study?

Notes on Key Terms and Formulas

  • Independent variable: manipulated by researcher; purposefully varied.

  • Dependent variable: measured outcome.

  • Operational definition: objective measurement criteria.

  • Confound: uncontrolled factor that can influence the dependent variable.

  • Random assignment: equal chance to be in any condition; improves internal validity.

  • Population vs. sample: population is the whole group of interest; sample is the subset studied.

  • Validity: study measures what it intends to measure.

    • Construct validity: the measure reflects the theoretical construct.
    • External validity: results generalize beyond the study.
    • Internal validity: observed effects are due to the manipulation, not confounds.

  • Reliability: measurements are consistent over time.

  • Central tendency:

    • Mean: xˉ=1n∑<em>i=1nx</em>i\bar{x} = \frac{1}{n}\sum<em>{i=1}^n x</em>i
    • Median: middle value (or average of two middle values if n is even).
    • Mode: most frequent value.

  • Variability:

    • Range: Range=max⁡(x)−min⁡(x)\text{Range} = \max(x) - \min(x)
    • Standard deviation (sample): s=1n−1∑<em>i=1n(x</em>i−xˉ)2s = \sqrt{\frac{1}{n-1}\sum<em>{i=1}^n (x</em>i - \bar{x})^2}

  • Correlation:

    • Pearson r: r=∑(x<em>i−xˉ)(y</em>i−yˉ)∑(x<em>i−xˉ)2  ∑(y</em>i−yˉ)2r = \frac{\sum (x<em>i - \bar{x})(y</em>i - \bar{y})}{\sqrt{\sum (x<em>i - \bar{x})^2}\; \sqrt{\sum (y</em>i - \bar{y})^2}}
    • Range of r: from −1.0-1.0 to +1.0+1.0 with ±1 indicating perfect correlation.

  • Meta-analysis: synthesis of results across multiple studies to strengthen evidence.

  • Ethical considerations summary:

    • Informed consent, deception and debriefing, risk-benefit analysis, anonymity/confidentiality, right to withdraw, IRB approval.