Statistical Thinking, Psychological Research Methods, and Research Designs

Statistical Thinking and Investigation Frameworks

• Basic Elements of a Statistical Investigation:

  • Planning the study: Identifying and formulating testable research questions.
  • Examining the data: Establishing appropriate methods to analyze the data, along with securing rigorous evidence for both reliability and validity.
  • Inferring from the data: Utilizing valid statistical methods to draw sound inferences.
  • Drawing conclusions: Assessing whether cause-and-effect conclusions can legitimately be drawn from the observed findings.

• $p$-Values and Significance Testing:

  • Definition of $p$-value: Indicates how often a random process would yield a result at least as extreme as what was obtained in the actual study, under the foundational assumption that there was nothing other than random chance at play.
  • Significance threshold: The $p$-value is typically evaluated against an established cut-off value, known as the level of significance, conventionally set at $0.05$.
  • Decision criteria: When the calculated $p$-value is smaller than the cut-off value ($p < 0.05$), the null hypothesis is rejected.

• Confidence Intervals:

  • Defined as an interval of plausible values for an unknown population parameter.
  • Operationally structured as the interval of values falling within the margin of error of a sample statistic.

• Random Sampling in Generalizing from Sample to Population:

  • Primary objective: Selecting a sample that is accurately representative of the entire population to permit generalizable conclusions.
  • Mechanism of random sampling: In its simplest form, random sampling ensures that every member of the population has an equal chance of being selected.
  • Implementation requirement: Generalizability holds if every member of the target population is randomly selected.

• Role of Random Assignment in Cause-and-Effect Conclusions:

  • Balances unobserved factors: Acts as a balancing mechanism across all variables that are not explicitly tracked or thought about.
  • Equalizes groups: Establishes an equal and similar distribution across experimental groups, making them as similar as possible in every aspect with the sole exception of the specific manipulation (such as the type of motivation applied).
  • Limitation: Random assignment does not always guarantee perfect equivalency across groups.

• Critique of Statistical Outcomes:

  • The influence of luck: Apparent findings can sometimes be attributable purely to random luck.
  • Imperfect randomness: Processes assumed to be stochastic may not always achieve genuine randomness in practice.

Psychological Research in Daily Life and Ecological Validity

• Limitations of Traditional Laboratory Experiments:

  • Methodological imbalance: Controlled laboratory studies tend to end up lopsided by maintaining high internal validity through careful control at the expense of generalizability.
  • Artificial parameters: Reveal what can happen under highly isolated, artificial conditions, but offer little insight into what actually does happen in natural settings.
  • Validity trade-offs: Researchers face continuous difficulties finding the balance between ensuring internal validity and establishing external validity.
  • Environmental divergence: Laboratory settings are typically unlike the settings encountered in real life ("IRL"), bringing into question the practical usefulness of laboratory findings.
  • Ecological validity defined: The degree to which an empirical effect has been obtained under conditions that are typical of what happens in everyday life.

• Contributions of Daily Life Research to Psychological Science:

  • Focus on natural experience: Investigates human experience and actions as "life as it's lived."
  • Wilhelm's methodology: Wilhelm has published a comprehensive "toolbox" designed to measure psychological experiences and overt behaviors within the context of daily life.
  • Real-time snapshots: Enables researchers to capture a detailed snapshot directly into people's ongoing lives.
  • Technological enablement: Recent developments in technology ("TECH") have made the widespread implementation of real-world daily life research possible.

• Methods for Conducting Real-World ("I.R.L.") Psychological Research:

  • Experience-Sampling Method (ESM): A collection of contemporary techniques designed to assess participants repeatedly in natural contexts.
  • Momentary self-report methods: Involves the administration of brief momentary questionnaires to collect in-the-moment responses, generating an immediate snapshot of what is going on at that specific time.
  • Day Reconstruction Method (DRM): Designed to obtain rich information about daily life without imposing the continuous burden of real-time momentary data collection; participants systematically reconstruct and report their specific experiences across a given day.
  • Direct behavioral observation: Involves capturing objective behavioral actions directly in real-world environments, such as through naturalistic video recordings.
  • Electronically Activated Recorder (EAR): An acoustic recording tool that collects intermittent samples of ambient sound bites, constructing an objective acoustic diary of participants' daily lives.
  • Ambulatory physiological monitoring: Tracking physiological reactions continuously as people go about their daily routines, including electrocardiograms (ECGs) and blood pressure (B.P.) measurements.
  • Ambulatory biological assessment: Collecting objective physiological data on fluctuating hormones and related biomarkers throughout the day.
  • Lived day analysis: An intensive observational methodology in which an active research team directly follows an individual throughout their day using a video camera.

Research Designs: Correlational versus Experimental Frameworks

• Differences Between Correlational and Experimental Research Designs:

  • Correlational design: Measures the statistical association between $2$ variables at a time, documenting how they naturally relate or go together without intervention.
  • Experimental design: Involves researchers actively manipulating or causing changes in the independent variable ($1$ variable at a time) and observing subsequent changes in the dependent variable.
  • Experimenter effects: Experimenters' personal expectations have the potential to influence the ultimate outcome of a study.
  • Bivariate patterns: Correlational research identifies patterns across $2$ variables at a time, but one cannot be certain about true relationships based on correlation alone.
  • Control of confounds: Experimental designs implement random assignment to actively balance extraneous participant characteristics and reduce confounding variables.

• Interpretation of Correlational Evidence:

  • Nature of observation: Scientists passively observe and measure naturally occurring phenomena without introducing external interventions or alterations.
  • Analytical boundaries: Useful for identifying patterns of association, but cannot be used to infer causal relationships between variables.

• Mechanisms of Causal Inference in Experiments:

  • Manipulation of variables: Experiments enable causal inference because researchers directly manipulate $1$ variable (the independent variable) and evaluate its measurable effects on another variable (the dependent variable).
  • Extraneous variable control: Researchers systematically control for other confounding factors, which allows them to definitively determine whether the independent variable was the actual cause of the observed change in the dependent variable.