PSY 2174 - Oct 21 - Comprehensive Study Notes on Research Methodology
Assignment Overview
Purpose: A short assignment to practice flexibility in research and create a positionality statement.
Focus: Based on individual's intersecting identities that influence their experiences and research approaches.
Objective: To get feedback and improve the assignment for the final decision proposal.
Positionality Statement
Definition: A reflective description of how one's personal identity impacts their approach to research.
Importance: Understanding how experiences and worldviews shape selection of research topics and questions.
Reflection Points:
Examine personal identities and how they relate to the research undertaken.
Identify biases and assumptions regarding the chosen topic.
Consider relationship with the population being studied and impact on participation.
Reflection on Identity
Encourage deep reflection on how identity intersects with research.
Impact on thought processes and question formulation.
Awareness of potential biases in approaching research subjects.
Understanding participant perspectives and enhancing rapport.
Resources Provided
A variety of resources for further reading and examples of positionality statements.
Emphasis on personalization of the statement based on individual experiences.
Example of a Positionality Statement
Researcher Experiences: Described by the professor through personal research experience.
Example: Interviewing parents and addressing potential discomfort regarding evaluation.
Approach: Fostering a learning environment where the researcher doesn’t position themselves as an expert.
Importance of framing: Reflects the intention to learn from participants rather than evaluate.
Assignment Details
Length Requirement: Minimum of 150 words, with an option to expand to up to 500 words.
Transition to Content Review
Transition back to the topic of reliability discussed in previous sessions.
Reliability vs. Validity
Key Concepts: Understanding how reliability and validity relate to measurement in research.
Reliability: Refers to the consistency of a measure across different instances.
Validity: Refers to whether the tool accurately measures the concept intended.
True Score Theory
Definition: Measurement reflects both a true value and an error component.
Components: Errors affecting measurement can be random or systematic.
Importance of estimating errors in observational research.
Types of Errors
Random Error:
Definition: Error caused by random factors affecting measurement inconsistently across subjects.
Effect: Increases data variability without shifting averages.
Example: Variability in test performance due to individual differences.
Systematic Error:
Definition: Error that consistently shifts measurements in one direction across a sample.
Effect: Influences the average performance in a measurable way.
Example: All participants influenced by a distracting environment in similar ways.
Addressing Measurement Errors
Strategies to mitigate errors:
Pilot testing instruments to gather preliminary feedback.
Training interviewers and coders to standardize data collection.
Double-checking data entry to reduce statistical errors.
Utilizing statistical procedures to adjust for errors where possible.
Employing triangulation by measuring constructs through multiple methods.
Types of Reliability Estimates
Inter-rater Reliability: Agreement between different observers/raters assessing the same phenomenon.
Test-Retest Reliability: Same test administered to the same sample at different times to check consistency.
Parallel Forms Reliability: Different versions of the same test administered to assess consistency across forms.
Internal Consistency: Used for multi-item scales to measure how well the individual items correlate with one another (Cronbach’s alpha).
Application in Proposals
Not required to calculate reliability estimates but can reference existing studies that report them for pre-existing measures.
Internal Validity
Definition: Relates to the accuracy of inferences regarding cause-effect or causal relationships in research.
Importance: Crucial for studies making causal claims on interventions or treatments.
Establishing Causation
Criteria for Establishing Causation:
Temporal Precedence: Cause must occur before effect.
Covariation: A relationship must exist between cause and effect (if A occurs, so does B).
Alternative Explanations: Ability to rule out other potential explanations for observed outcomes.
Challenges in Establishing Causality
Difficulty in eliminating alternative variables and confounding factors.
Importance of well-designed research that effectively controls for potential confounds.
Threats to Internal Validity
Identifying threats in single group designs:
History Threat: External event coinciding with the timing of the study impacting results.
Maturation Threat: Changes in participants over time affecting results independently of treatment.
Testing Threat: Previous exposure influencing performance on post-test.
Instrumentation Threat: Variability in measurement tools affecting reliability.
Mortality Threat: Loss of participants impacting research outcomes.
Regression to the Mean: Extreme results can regress toward the average due to random chance.
Mitigation Strategies
Use of control groups to compare against treatment groups and verify that observed changes are due to interventions.
Random assignment of participants to minimize selection bias and ensure comparable groups.
Social Interaction Threats to Validity
Awareness of groups knowing their treatment/control status, leading to biases in outcomes.
Example: Participants adjusting behavior because they are aware they are in a study context.
Strategies for Reducing Bias
Implementation of double-blind study designs where both researchers and participants are unaware of treatment assignments.
Wrap-Up
Emphasis on understanding threats to internal validity in both single and multiple group studies.
Importance of rigorous research design and consideration of methodology to ensure reliable, valid results.