Stats - Sampling & Quantitative Data Collection Methods

WEEK 5: SAMPLING & QUANTITATIVE DATA COLLECTION METHODS

Presenter: Dr. Katie Gilligan-Lee

Course: PSY 10100: Research Methods and Statistics I

Learning Objectives

  • Part 1: Sampling (Qualitative & Quantitative)

  • Part 2: Quantitative Data Collection Methods: Surveys & Questionnaires

  • Part 3: Designing Questionnaires

  • Part 4: Mode of Delivery

Reading for the Week

  • Essential reading from core text:

    • Shaughnessy, Zeichmeister and Zeichmeister – Chapter 5: Survey Research

Part 1: Sampling

How Do We Collect Research Data?

  • Techniques for Data Collection:

    • Range from qualitative to quantitative

    • Includes experimental designs for objective data (time, scores, biofeedback)

    • Quantitative = surveys and questionnaires

    • Qualitative = interviews and focus groups

    • Observational methods are used in both

  • Importance of Sampling: Before data collection, sampling must be considered.

Definitions

  • Population: Entire collection about which information is desired (e.g., adults over 60, male smokers)

  • Sample: A subset selected from the population for study

    • Advantages: Cost-effective, time-saving, reliability

    • Disadvantages: Sampling process issues, representativeness

Sample Size

  • Number of participants affects power of study

  • Size can be independent of population size based on resources and accessibility

  • In quantitative designs, sample size can be calculated to ensure study power

Effect Sizes and Statistical Power

Effect Sizes

  • Benchmarks for assessing the size of effects (differences or associations)

  • Larger effect sizes make it easier to detect changes

Statistical Power

  • Defined as: Probability of correctly rejecting the null hypothesis

  • Small sample sizes may not reliably detect effects unless large

  • Tools like Gpower can assist in calculating needed sample sizes

Sample Biases

Types of Biases

  1. Selection Bias: Systematic differences in sample

    • Inadequate sampling frame

    • Convenience sampling limitations (e.g., UCD students not representative)

  2. Non-Response Bias: Refusal or attrition from sample

    • Impact of volunteer characteristics compared to non-volunteers

Sampling Methods

Types of Sampling

  • Probability Sampling: Considered gold standard in quantitative research, all members have an equal chance of selection

    • Simple Random Sampling (SRS): All individuals have equal selection chance

    • Systematic Sampling: Structured system starting with random selection

    • Stratified Random Sampling: Divided into strata, random selection within strata

    • Cluster Sampling: Population divided into clusters, clusters randomly selected, and all in cluster included

Part 2: Quantitative Methods

Quantitative Data Collection Methods

  • Physiological Measurement: EEG, skin conductance, heart rate, blood pressure

  • Performance on Tasks: Operational definition of concepts through task performance

  • Questionnaires: Used to quantify a wide range of concepts

  • Surveys: Capture detailed experiences and opinions

Surveys & Questionnaires

  • Questionnaires assess specific concepts and provide scores

    • E.g., Strengths and Difficulties Questionnaire includes overall scores and subscales

  • Quantitative research relies on valid and reliable measures

Psychometric Properties

Key Aspects

  • Reliability: Consistency of scores

    • Internal Reliability: Consistent high scores across items

    • Test-Retest Reliability: Similar scores over time

  • Validity: Measures what intended

    • Face Validity: Items assess the intended construct

    • Discriminant Validity: Differentiates between groups

    • Convergent Validity: Consistent with similar tests

    • Divergent Validity: Weak correlations with unrelated constructs

Benefits of Questionnaire Research

  • Efficient, low-cost data collection

  • Variety of question types

  • High generalizability from large sample sizes

Part 3: Designing Questionnaires

Considerations for Design

  1. Use of Likert scales

  2. Deciding on open vs. closed questions

  3. Include reverse coded items

  4. Self-report vs. informant report

  5. Avoid confusing phrasing

  6. Minimize risk of bias

  7. Avoid making agreement/disagreement responses undesirable

  8. Minimize fence-sitting and floating

  9. Pretest questions

  10. Question order impacts results

1) Likert Scales

  • Participants rate responses numerically

  • Can have equal/unequal response options

  • Example anchors: Strongly Agree to Strongly Disagree

2) Open vs. Closed Questions

  • Open: Full range of responses, qualitative insights

  • Closed: Limits ambiguity, quicker to code, but may exclude unexpected responses

3) Reverse Coded Items

  • Ensure that higher scores consistently indicate the same construct

4) Self-Report vs. Informant Report

  • Self-report: Individual responses

  • Informant report: Responses about another person

5) Avoid Confusing Phrasing

  • Use simple language, avoid double negatives, and complexity

6) Minimize Risk of Bias

  • Balance response options, avoid emotionally loaded questions

7) Avoid Disagreeable Language

  • Ensure neutral framing of questions

8) Address Fence-Sitting and Floating

  • Allow "don't know" options judiciously

9) Pretest Your Questions

  • Utilize cognitive interviewing and follow-ups to evaluate comprehension

10) Question Order

  • Consider context effects in the sequence of questions

Part 4: Mode of Delivery

Modes of Administration

  1. Group Administered

  2. Mailed/Posted

  3. Electronic/Web-Based

  4. Phone Survey

  5. In-Person Interview

1. Group-Administered

  • Self-administered in group settings, efficient but audience may not be captive

2. Mailed Surveys

  • Self-administered at home, requires follow-ups and may involve postage costs

3. Electronic Surveys

  • Online, low cost, easy to develop, can be longer

4. Phone Surveys

  • Administered by professionals, lower response rates occur

5. In-Person Interviews

  • High response rates, comprehensive but costly and time-consuming

Reminder & Task

  • Next Week: Research Ethics & Exam Information, practice MCQ test

  • Task Before Next Class: Find a questionnaire not covered in class, submit details on Brightspace by deadline.

Questions?

  • Additional questions may be asked via the discussion board on Brightspace.