Evaluating Samples and Research Designs

Sample Evaluation and Representativity

  • Sample Quality and Representativity:

    • The quality of any research sample is determined by its representativity.
    • Representative Sample: A sample in which the characteristics of the sample precisely match the characteristics of the target population, differing only in size (characteristics of sample=characteristics of population\text{characteristics of sample} = \text{characteristics of population} except for size).
  • Demonstration: Psychology Students Population vs. Representative Sample:

    • Population Distribution:
    • Gender: 20%20\% men, 75%75\% women, 5%5\% other.
    • Track: 60%60\% bilingual, 40%40\% English.
    • Representative Sample (N=100N = 100 participants):
    • Gender: 2020 men, 7575 women, 55 other.
    • Track: 6060 bilingual, 4040 English.
  • Sampling Methods and Representativity:

    • Probability Sampling: The only methodological approach that ensures a representative sample can be obtained.
    • Non-Probability Sampling: Introduces substantial risk of the sample becoming distorted or systematically biased.

Cartoon illustrating sampling bias where a survey asking about liking surveys yields distorted results

Potential Sample Distortions: Sampling Bias and Response Bias

  • Sampling Bias:

    • Occurs when there is an underrepresentation or overrepresentation of a specific segment of the population within the sample.
  • Response Bias and Response Rate:

    • Response Rate: The percentage of individuals originally selected for the sample who actually agree to participate and complete the study.
    • Core Evaluative Question: Are the respondents comparable to the non-respondents? If individuals who choose to participate systematically differ in views, behaviors, or traits from those who decline, response bias occurs.
  • Empirical Case Study: Shere Hite (1987):

    • Reference: Hites, S. (1987). Woman and love: A cultural revolution in progress. New York: Knopf.
    • Methodology:
    • Target sample approached: 100,000100{,}000 women contacted.
    • Format: Questionnaire study.
    • Completed responses received: 4,5004{,}500 respondents.
    • Calculated Response Rate: 4,500100,000×100=4.5%\frac{4{,}500}{100{,}000} \times 100 = 4.5\%.
    • Reported Findings:
    • 70%70\% of respondents reported having engaged in an extra-marital affair.
    • 95%95\% felt emotionally abused by their partner.
    • Methodological Limitation:
    • With a response rate of only 4.5%4.5\%, severe response bias is present; the women who chose to complete and return the questionnaire likely held stronger grievances, negative experiences, or differing relationship behaviors compared to the 95.5%95.5\% of women who did not respond.

Descriptive Research and Empirical Domains

  • Definition of Descriptive Research:

    • Research designed specifically to provide an accurate, systematic description of the behaviors, thoughts, or feelings of a particular group or population.
  • Survey Studies:

    • Involve soliciting self-reported information regarding attitudes, convictions, opinions, political leanings, and related constructs.
    • Often used for public opinion polling and political seat projections.

Dutch House of Representatives seat projection comparing election results and polling dates with error margins

  • Political Seat Projection Details (Dutch House of Representatives / Tweede Kamer):

    • Compares historical general election results (TK23) with successive polling snapshots (17-feb-25 and 17-mrt-25), incorporating confidence intervals/error margins across political parties:

      • PVV: 3737 seats (TK23), 3434 seats (17-feb-25), 3030 seats (17-mrt-25).
      • GL-PvdA: 2525 seats (TK23), 2424 seats (17-feb-25), 2727 seats (17-mrt-25).
      • VVD: 2424 seats (TK23), 2020 seats (17-feb-25), 2525 seats (17-mrt-25).
      • CDA: 55 seats (TK23), 1616 seats (17-feb-25), 1818 seats (17-mrt-25).
      • D66: 99 seats (TK23), 1111 seats (17-feb-25), 1111 seats (17-mrt-25).
      • PvdD: 33 seats (TK23), 88 seats (17-feb-25), 77 seats (17-mrt-25).
      • SP: 55 seats (TK23), 88 seats (17-feb-25), 66 seats (17-mrt-25).
      • DENK: 33 seats (TK23), 33 seats (17-feb-25), 44 seats (17-mrt-25).
      • CU: 33 seats (TK23), 33 seats (17-feb-25), 44 seats (17-mrt-25).
      • JA21: 11 seat (TK23), 33 seats (17-feb-25), 44 seats (17-mrt-25).
      • Volt: 22 seats (TK23), 55 seats (17-feb-25), 33 seats (17-mrt-25).
      • BBB: 77 seats (TK23), 44 seats (17-feb-25), 33 seats (17-mrt-25).
      • FvD: 33 seats (TK23), 44 seats (17-feb-25), 33 seats (17-mrt-25).
      • SGP: 33 seats (TK23), 44 seats (17-feb-25), 33 seats (17-mrt-25).
      • NSC: 2020 seats (TK23), 33 seats (17-feb-25), 22 seats (17-mrt-25).
    • Demographic Research:

  • Focuses on quantifying and describing elementary life events, demographic transitions, and population milestones (e.g., birth rates, fertility, marriage rates, mortality, age milestones).

Graph showing teen pregnancy incidence rising through adolescence and dropping to zero at age 20

  • Definitional Fallacies in Demographics:

    • Data tracking teen pregnancy rates illustrates how classification definitions dictate distributions: pregnancies rise through the teen years (ages 1313 through 1919, peaking around age 1919), before dropping precipitously to zero at age 2020, which occurs purely because individuals aged 2020 and older are no longer categorized as teenagers.

    • Epidemiological Research:

  • Gathers empirical data on the occurrence, prevalence, and incidence of diseases, mental disorders, and physical health conditions within populations.

Bar chart of sleeping problems and sleep deprivation rates across age groups and gender for 2022

  • CBS Health Data on Sleep (2022):
    • Evaluates individuals affected by sleep deprivation and suffering from general sleeping problems across cohorts (12 to 17 yrs12\text{ to }17\text{ yrs}, 18 to 24 yrs18\text{ to }24\text{ yrs}, and total population) categorized by gender (Total, Female, Male):
      • Total Population:
      • Affected by sleep deprivation: Approximately 7%7\% (12 to 17 yrs12\text{ to }17\text{ yrs}), 13%13\% (18 to 24 yrs18\text{ to }24\text{ yrs}), and 11%11\% (Total).
      • Sleeping problems: Approximately 17%17\% (12 to 17 yrs12\text{ to }17\text{ yrs}), 26%26\% (18 to 24 yrs18\text{ to }24\text{ yrs}), and 22%22\% (Total).
      • Females:
      • Affected by sleep deprivation: Approximately 9.5%9.5\% (12 to 17 yrs12\text{ to }17\text{ yrs}), 15%15\% (18 to 24 yrs18\text{ to }24\text{ yrs}), and 13%13\% (Total).
      • Sleeping problems: Approximately 20%20\% (12 to 17 yrs12\text{ to }17\text{ yrs}), 27.5%27.5\% (18 to 24 yrs18\text{ to }24\text{ yrs}), and 24.5%24.5\% (Total).
      • Males:
      • Affected by sleep deprivation: Approximately 5%5\% (12 to 17 yrs12\text{ to }17\text{ yrs}), 11.5%11.5\% (18 to 24 yrs18\text{ to }24\text{ yrs}), and 8.5%8.5\% (Total).
      • Sleeping problems: Approximately 14%14\% (12 to 17 yrs12\text{ to }17\text{ yrs}), 25%25\% (18 to 24 yrs18\text{ to }24\text{ yrs}), and 20%20\% (Total).

Explanatory Research in Psychology

  • Descriptive vs. Explanatory Focus:

    • Most research within psychological science is non-descriptive.
    • Rather than aiming solely to catalog population parameters, psychological investigations predominantly seek to explain functional and causal relationships between variables.
  • Three Primary Pillars of Explanatory Psychological Science:

    • 1. Hypothesis Testing: Formulating theoretical expectations regarding how variables interact and testing them systematically against empirical observations.
    • 2. Replications: Conducting repeated studies across diverse conditions and contexts to confirm whether observed effects and relationships remain stable.
    • 3. Universal Phenomena: Investigating basic cognitive, neurological, affective, and behavioral mechanisms presumed to operate universally across human populations, reducing the absolute necessity for representative demographic matching when examining universal psychological architecture.