Research Methods: Observation, Bias, Reliability & Validity

Research

A systematic, intentional, and iterative process of building knowledge about the world.

Research vs. everyday thinking

Research uses systematic methods, evidence, and critical analysis to minimize biases.

Qualitative research

Research that explores meanings and experiences using observations, words, and narratives.

Quantitative research

Research that uses numerical measurements and statistical analysis to test hypotheses.

Mixed methods

Research combining qualitative and quantitative approaches.

Inductive reasoning

Bottom-up reasoning that uses observations and patterns to develop theories.

Deductive reasoning

Top-down reasoning that starts with a theory and tests predictions using data.

Research cycle

Theories lead to hypotheses, which are tested using observations that can support or revise theories.

Theory

A model or explanation of how the world works.

Hypothesis

A specific, testable, and falsifiable prediction.

Falsifiability

A hypothesis must be capable of being disproven by evidence.

Exploratory research

Research that investigates a topic to discover patterns or generate hypotheses.

Positionality

A researcher's personal identities, beliefs, experiences, and judgments.

Reflexivity

Reflecting on how one's positionality influences the research process.

Availability heuristic

Relying on information that comes easily to mind.

Recognition heuristic

Preferring a familiar option over an unfamiliar one.

Confirmation bias

Favoring information that supports existing beliefs.

Anchoring bias

Overreliance on initial information or impressions.

Framing effect

The way information is presented influences judgments.

Cognitive bias

A systematic distortion in thinking or judgment.

Small worlds

Specific social groups or settings studied in depth in qualitative research.

Thick description

Detailed description of behavior and its social or cultural context.

Thin description

Description of observable behavior without extensive contextual interpretation.

Generalizability

The extent to which findings from a sample apply to a larger population.

Experimental research

Research in which investigators manipulate variables to test their effects.

Correlational research

Research examining associations between variables without manipulating them.

Observational research

Systematically recording behavior or events without necessarily manipulating variables.

Naturalistic observation

Observing behavior in its ordinary environment.

Controlled observation

Observing behavior in a setting or situation arranged by the researcher.

Participant observation

Research in which the investigator participates in the group or activity being studied.

Public observation

Observing behavior in a setting accessible to the public.

Field notes

Detailed records of observations, context, and relevant events.

Audit trail

Documentation of how data were collected, coded, and analyzed.

Qualitative observational data

Non-numerical descriptions of behavior, interactions, and context.

Quantifying observations

Converting observed behaviors into counts, frequencies, or percentages.

Reliability

The consistency of measurements or research findings.

Validity

The extent to which a method measures or captures what it intends to study.

Inter-rater reliability

The extent to which different observers agree in their ratings or coding.

Percent agreement

The percentage of observations on which raters agree.

Cohen's kappa

A measure of agreement between raters that accounts for chance agreement.

Pearson's r

A correlation coefficient measuring the strength and direction of a linear relationship.

Researcher triangulation

Using multiple researchers to compare observations or interpretations.

Data triangulation

Using multiple participants, sources, or measurements to examine a phenomenon.

Methodological triangulation

Using multiple research methods to study the same phenomenon.

Member checking

Asking participants to verify the accuracy of transcripts, interpretations, or findings.

Replication

Repeating a study to assess whether findings are consistent.

Observer training

Preparing observers to record or code behavior consistently.

Trustworthiness

The credibility and rigor of qualitative research findings.

Gamma or Yule's Q

A measure of association used in the class example to examine whether behaviors occur together.

Positive association

Two behaviors or variables tend to occur together.

Negative association

As one behavior or variable occurs more, the other tends to occur less.

Correlation vs. causation

An association between variables does not establish that one causes the other.

Geller et al. (1986)

Naturalistic observations of beer drinking among college students.

Kraut and Johnston (1979)

Observational study examining smiling and its social and emotional associations.

Levine and Norenzayan (1999)

Study comparing the pace of life across 31 countries.

Observational study with statistical analysis

Research that systematically observes behavior and quantitatively analyzes the resulting data.

Research ethics

Principles governing responsible research and protection of participants.

Nuremberg Code (1947)

Ethical code emphasizing voluntary consent, minimizing harm, and the right to withdraw.

Declaration of Helsinki (1964)

Ethical principles for medical research involving human participants.

Tuskegee syphilis study

Unethical study involving deception and withholding treatment that contributed to reforms in research ethics.

National Research Act (1974)

US legislation establishing a framework for protecting human research participants and IRB oversight.

Belmont Report

Ethical framework emphasizing respect for persons, beneficence, and justice.

Respect for persons

Respecting autonomy and protecting individuals with diminished autonomy.

Beneficence

Maximizing possible benefits while minimizing potential harms.

Justice

Fair distribution of the benefits and burdens of research.

Institutional Review Board (IRB)

Committee that reviews research involving human participants to protect their rights and welfare.

Informed consent

Voluntary agreement to participate after receiving relevant information about a study.

Voluntary participation

Participants choose freely whether to participate without coercion.

Right to withdraw

Participants can stop participating without penalty.

Coercion

Using threats or pressure to influence participation.

Deception

Deliberately withholding or misrepresenting information about a study.

Debriefing

Explaining the study to participants afterward, especially when deception was used.

Confidentiality

Researchers know participants' identities but protect their information from disclosure.

Anonymity

Participants' identities are not known or linked to their responses.

Privacy

A person's control over access to themselves and their personal information.

Risk-benefit assessment

Evaluating whether potential research benefits justify the risks to participants.

Vulnerable populations

Groups needing additional protections because of increased risk or reduced ability to consent.

Research integrity

Conducting and reporting research honestly and responsibly.

Fabrication

Inventing research data or results.

Plagiarism

Presenting another person's work or ideas as one's own.

Scientific misconduct

Serious violations of responsible research practices, including fabrication and plagiarism.

IRB approval

Ethical review and authorization required for studies subject to IRB oversight before research begins.

Interview research

Collecting detailed self-reported experiences and perspectives through purposeful conversations.

Purpose of qualitative interviews

Understanding participants' experiences, meanings, and perspectives in depth.

Interview research approach

Generally qualitative, exploratory, and inductive.

Interview data

Narratives, stories, experiences, and spoken responses.

Structured interview

Interview using standardized questions in a predetermined order.

Semi-structured interview

Interview guided by prepared questions while allowing follow-up questions.

Unstructured interview

Flexible conversation with few predetermined questions.

Open-ended question

Question allowing participants to answer in their own words.

Closed-ended question

Question restricting responses to predetermined options or short answers.

Rapport

A trusting and comfortable relationship between interviewer and participant.

Probing question

Follow-up question that encourages elaboration or clarification.

Leading question

Question worded to suggest or encourage a particular response.

Acquiescence bias

Tendency to agree with statements or questions regardless of actual beliefs.

Double-barreled question

A question asking about two different things at once.

Interview sampling

Selecting participants who can provide relevant information about the research topic.

Snowball sampling

Recruiting additional participants through referrals from existing participants.

Non-probability sampling

Sampling in which individuals do not have a known probability of selection.

Interviewer positionality

How the interviewer's background, beliefs, and experiences may affect the interview.