Introduction to Psychological Research Methods Flashcards
Descriptive Research and Inter-Rater Reliability
Descriptive Research Overview:
Descriptive research focuses on documenting and measuring phenomena as they exist naturally without manipulating independent variables or conducting complex inferential statistical tests.
It seeks to address questions such as: "What did participants state when asked specific questions?" or "What is the observed frequency or occurrence rate of a specific phenomenon?"
Subjectivity and Bias in Observation:
Human observers are inherently subject to cognitive bias, perceptual limitations, and error.
A study relying on a single individual's observation or interpretation lacks scientific validity.
Establishing that multiple independent observers agree on what was read, heard, or observed provides essential construct and interpretation validity to the findings and results section of a research report.
Qualitative Research Coding Workflow:
1. Collect qualitative, open-ended narrative responses from participants using a structured or semi-structured list of questions.
2. Multiple researchers meet prior to data analysis to discuss, define, and establish expected or potential thematic categories.
3. Coding Process: Researchers independently read a sub-sample of responses, extract specific text segments sharing identical thematic ideas, and attach qualitative labels to those segments.
4. Researchers reconvene to systematically assess and verify whether they arrived at identical qualitative labels for the same text segments.
5. Upon reaching a consensus framework, researchers apply the labels to an expanded batch of participant responses.
6. Specialized software computes an quantitative Inter-Rater Reliability Score to confirm that coding agreement meets formal statistical criteria.
7. The agreement score must reach a pre-specified threshold (categorized as being "in the green") before researchers can proceed independently to code the remaining dataset.
Observational Research Example:
Behavioral operational definitions must be concretely established prior to initiating data collection.
The observer systematically monitors, identifies, and records target behaviors in real time.
The precision and accuracy of the primary observer are formally validated through inter-rater reliability assessments.
Correlational Methodology and Variable Operationalization
The Correlational Method:
Evaluates the mathematical relationship between two or more co-occurring variables without inferring direct cause-and-effect mechanisms.
Spurious Correlations and Confounding Variables:
Classic Spurious Example: The observed statistical alignment between rising ice cream sales and elevated homicide/crime rates.
Third Variable (Confound): Ambient temperature (heat).
Causal Mechanism: Warm weather causes an increase in outdoor social density, frustration, and irritability, which independently increases violent crime rates; elevated outdoor heat simultaneously increases consumer demand for ice cream.
Conceptual Distinction: Two variables sharing a statistical relationship driven by a mutual third variable (an exogenous factor) are connected only through that shared connection, not through direct causality.
Operational Definitions:
The precise, concrete procedure or instrument used to quantify a construct is critical to evaluating correlational outcomes.
Inappropriate operationalization can invert or obscure true statistical relationships.
Operationalizing Depression vs. Activity:
Measuring Activity via Step Counts: Utilizing digital step counters tracking total physical steps taken over a defined timeframe (e.g., ).
Measuring Depression: Administering a psychometrically validated, highly reliable depression scale where higher quantitative scores denote higher severity of depressive symptoms.
Demographic Variables & Moderators: Collecting demographic data (e.g., educational background, racial identity, gender identity, age) to identify potential moderating variables that alter the strength or direction of the primary correlation.
Exclusion Criteria: Identifying confounding physical factors (e.g., physical mobility impairments, motor disabilities) that lower step count independently of psychological depression.
Types of Correlation: Positive vs. Negative
Negative Correlation:
Occurs when two variables move in opposite directions (as one variable increases, the other decreases).
Hypothesis Example: High depression scores correlate statistically with low step counts.
Interpretation: A negative sign denotes directionality and opposition, not a bad or weak relationship. It indicates that the constructs are opposing yet significantly linked.
Positive Correlation:
Occurs when two variables move in the same direction simultaneously (as one variable increases, the other increases; as one variable decreases, the other decreases).
Hypothesis Example: Re-operationalizing activity as time spent sleeping or hours staying at home. As depression severity increases, total hours spent sleeping also increases.
Interpretation: A positive sign denotes directional alignment across both high and low extremes.
Experimental Design and Random Assignment
Experimental Manipulation:
Experiments require active manipulation of an independent (causal) variable by creating distinct conditions, doses, or operational levels.
Random Assignment:
The procedure of placing participants into experimental treatment groups purely by chance, ensuring every participant has an equal probability of assignment to any condition.
Eliminates baseline systematic differences between treatment groups.
Flawed Experimental Design & Geographic Confound Scenario:
Flawed Setup: Investigating the effect of depression on sleep duration by non-randomly assigning participants based on geographical regions: Ohio (assigned severe depression), Minnesota (assigned moderate depression), and Western North Carolina (assigned mild depression).
Systematic Confounders in Ohio: Limited winter sunlight, freezing ambient temperatures, regional socio-cultural drinking practices ("Midwest social activities"), and localized psychological distress connected to regional sports franchises (e.g., Cleveland Browns losses).
Methodological Failure: Baseline unmeasured environmental confounds (lack of sun, cold climate, alcohol consumption) directly impact sleep patterns, destroying internal validity and invalidating assertions that depression manipulation caused sleep variances.
Random Selection, Sampling, and Triangulation
Random Selection:
The process of drawing participants from a target population such that every individual in that population has an equal chance of inclusion in the study sample.
Sampling Limitations in Psychological Science:
Heavy institutional reliance on university undergraduate student samples due to recruitment accessibility.
Sample Characteristics: High average educational background, higher relative socio-economic standing, and restricted young-adult age distribution.
Generalizability Threats: Findings derived exclusively from university students may fail to extrapolate to broader populations, particularly for variables influenced by age-dependent physiological or social changes (e.g., mobility and step count trends in older demographics).
Methodological Triangulation:
The application of multiple operational measures or distinct research methodologies to evaluate a single construct.
Example (Activity Measures): Combining step counters, daily home-leaving frequency, phone/text social communication logs, and biometric ring sensors (e.g., Oura Ring).
Analytical Advantage: Demonstrating consistent relationships across diverse metrics increases statistical and methodological confidence, revealing nuanced domain-specific patterns (e.g., depression correlating with physical activity measures but not social engagement logs).
Statistical Significance and Statistical Power
Understanding the $p$-Value:
The probability value () represents the probability that observed experimental results occurred purely due to random chance or sampling error.
Significance Thresholds:
The standard scientific criterion for statistical significance is an alpha level of ( or lower probability that results occurred by chance).
If , the result is non-significant, and researchers cannot assert a true experimental effect or correlation.
Factors Causing False Non-Significance (Type II Errors):
Inadequate Measurement: Insufficiently sensitive or unreliable measurement instruments.
Inadequate Statistical Power: Insufficient sample sizes (e.g., using participants per condition instead of participants per condition).
Research Validity: Internal vs. External
Internal Validity:
The degree to which an experimental design accurately isolates the independent variable without extraneous noise, bias, or uncontrolled confounding variables driving the outcome ("trash in, trash out").
Extraneous Variables to Control in Depression/Sleep Studies:
Pharmaceutical prescriptions (e.g., active sleep aids, psychiatric medications).
Pre-existing clinical disorders (e.g., chronic insomnia, generalized anxiety disorder).
Seasonal timing and climate changes (e.g., Seasonal Affective Disorder during winter months).
Employment structure (e.g., night shift or third shift work schedules).
External Validity and Generalizability:
External Validity: The extent to which experimental results reliably reflect ecological real-world behavior across real settings, diverse contexts, and broad populations.
Generalizability: Extent to which findings apply beyond specific university lab spaces or narrow subject demographics.
Limitations Section: A mandatory section in scholarly peer-reviewed papers where researchers document study weaknesses, uncontrolled extraneous factors, sample constraints, and guidelines for future replication.
Psychological Realism and Measurement Challenges
Psychological Realism:
The degree to which the psychological processes triggered during an experimental manipulation match the true, natural psychological experiences encountered in everyday life.
If participants realize they are in an artificial lab setup and fail to register authentic emotion (e.g., true social embarrassment), the study lacks psychological realism.
Operationalizing Subjective Psychological Constructs (e.g., Embarrassment):
Physiological Indicators: Elevated heart rate, facial flushing/vasodilation (subject to confounds like baseline dermatological conditions).
Behavioral Indicators: Postural adjustment, collar-tugging, somatic fidgeting.
Composite Operationalization: Requiring participants to satisfy a multi-faceted threshold (e.g., requiring concurrent criteria: facial redness, elevated heart rate, and observed fidgeting).
Phenomenological Nature: Acute embarrassment alters subjective temporal perception (e.g., feeling trapped in "slow motion"). Pop-culture portrayals (e.g., Tim Robinson's performance in The Chair Company or I Think You Should Leave, such as the scene involving a slow-motion public fall from a broken stage chair) illustrate the severe cognitive and physical distress associated with true embarrassment.
Validity Tensions and Objectivity Threats
Retrospective Self-Report Bias:
Asking participants to quantify emotional states after learning the study's intent yields defensive, inaccurate, or post-hoc modified responses.
Fatigue Factor:
Deterioration in participant accuracy or effort occurring during lengthy testing sessions.
High-priority dependent measures placed exclusively at the end of exhaustive surveys risk corrupted data due to participant fatigue or rushing.
Validity Tensions:
Increasing internal validity through strict laboratory controls often reduces external validity and psychological realism.
Replication, Meta-Analysis, and Cross-Cultural Research
Replication:
Re-running experimental protocols with new independent samples and contexts to verify the robustness of original conclusions.
Meta-Analysis:
A statistical methodology that aggregates and synthesizes effect sizes across hundreds or thousands of prior independent studies examining the same research question.
Computes an overarching aggregate effect size that balances localized internal validity flaws in single studies.
Effect Size vs. Statistical Significance:
Statistical significance confirms whether an effect exists beyond chance; Effect Size quantifies the magnitude or practical real-world value of that effect.
Large sample sizes can yield statistical significance () for trivial real-world differences (e.g., spending or for an extra or of sleep).
Cross-Cultural Psychological Research:
Evaluates whether psychological constructs, baseline norms, and clinical manifestations hold across global cultures.
Schizophrenia Symptomatology Variations: In the United States, auditory hallucinations frequently present with aggressive, distressful, or violent content. In Eastern nations and Britain, auditory hallucinations often present as familiar, supportive voices delivering routine directives (e.g., instructions to brush teeth or practice self-care).
Socio-Cultural Norms: Individualistic values ("pulling oneself up by bootstraps") influence baseline behaviors differently than collectivist norms.
Micro-Cultural Contexts: Cultural shifts occur micro-regionally across states or neighboring municipalities (e.g., Eastern North Carolina vs. Western North Carolina; Hendersonville vs. Asheville cultural variances impacting localized behavioral outcomes like matcha consumption rates).
Ethical Considerations and the Institutional Review Board (IRB)
Core Ethical Imperative:
Scientific curiosity or theoretical value never justifies inflicting psychological or physical harm on human participants.
Institutional Review Board (IRB):
An administrative and ethics oversight committee charged with screening, modifying, approving, or rejecting research protocols involving human subjects.
Roughly general population metrics contextualize societal psychological trends requiring IRB protections.
Informed Consent Requirements:
Participants must receive written documentation outlining general study parameters, financial compensation, explicit voluntary participation protections, right to withdraw at any time without penalty, and data confidentiality/security protocols prior to signing.
Study details may remain intentionally general to prevent demand characteristics from corrupting natural responses.
Deception and Debriefing Protocols:
Deception: Concealing the precise experimental hypothesis or using confederates to preserve psychological realism must be explicitly justified to and approved by the IRB.
Debriefing: Mandatory post-experimental protocol where researchers fully disclose all hidden study objectives, reveal any deceptive elements, address participant concerns, and supply professional mental health resources.