Paper 3 research methods vocabulary

Generalizability

is the extent to which results of the study can be applied to the broader context - beyond the sample and the settings used in the study itself.

Population validity

the extent to which findings can be generalized from the sample to the target population.

Ecological validity

refers to the extent to which findings can be generalized from the experiment to other settings or situations.

Temporal validity

an external validity that refers to the generalizability of a study’s results across time.

Construct validity

the extent to which results of the study can be generalized from operationalizations to theoretical constructs.

Transferability

is the extent to which we can transfer the findings from one study to another context. It is the qualitative research equivalent of generalizability and means the same thing. We do not use the term generalizability in qualitative research because the sampling methods are not representational, so the findings are never ‘generalized’.

Case to case generalization

which refers to applying the findings from the setting of the research study to other similar settings.

Theoretical generalization

generalization is made from observations to a broader theory. Theory plays a much greater role in qualitative research than quantitative.

Sample to population generalization

refers to applying the results of the study from the participants who took part in the study to the wider population.

Research bias

results from any deviation from the truth, causing distorted results and wrong conclusions. Bias can occur at any phase of your research, including during data collection, data analysis, interpretation, or publication. Research bias can occur in both qualitative and quantitative research. Understanding research bias is important for several reasons.

Researcher triangulation

Researcher triangulation involves multiple independent researchers analyzing the same data to ensure objectivity. Instead of relying on a single person's interpretations, different analysts provide insights, which can then be compared for consistency. This approach prevents bias from arising due to subjective viewpoints.

Double-blind controls

methodological safeguards where both participants and experimenters are unaware of key experimental conditions. This prevents either party from unintentionally influencing results due to expectations or subconscious cues.

Replication

the process of repeating an experiment with different samples, researchers, and contexts to verify the reliability of findings. A study with strong scientific integrity should produce similar results when replicated.

Random sampling

Random sampling ensures that every individual in the population has an equal chance of being selected, preventing systematic biases in sample recruitment. Stratified sampling divides participants into key subgroups (e.g., age, gender) to maintain balanced representation.

Transparent reporting

Transparent reporting ensures that all study findings—including non-significant results—are documented in publications. This prevents the selective omission of data that may not support the researcher’s hypothesis.

Deception

misleading participants about the true purpose of the study to prevent them from consciously adjusting their behavior. This technique ensures their actions remain natural and uninfluenced by expectations.

Anonymity

Anonymous data collection ensures that participants’ responses are not personally identifiable, reducing social pressure. Indirect questioning

involves asking about general behaviors or peers' actions rather than personal ones, encouraging honest responses.

Hawthorne effect

–participants modify behavior simply because they know they are being watched.

By observing participants covertly or in familiar settings, researchers prevent behavioral adjustments that occur due to awareness of observation, ensuring more natural responses.

Acquiescence bias

When participants tend to agree with statements regardless of their true beliefs.

Participants must actively engage with reverse-coded questions, preventing them from defaulting to agreement without properly considering the statement’s meaning.

Single-blind control

ensures that participants do not know which condition they have been assigned to, preventing them from modifying their behavior based on perceived expectations. Researchers, however, remain aware of the conditions.

Counterbalancing

ensures that order effects—where condition sequence affects responses—are controlled by altering test order across participants.

Random allocation

ensures participants are equally distributed across experimental conditions, preventing systematic differences between groups. Matching techniques involve pairing participants with similar characteristics (e.g., age, intelligence) across conditions.

Repeated measures design

means the same participants experience all experimental conditions, reducing individual differences as a confounding factor.

Curvilinear relationship

In calculating the correlation between two variables, we assume the relationship between them is linear when there may be a hidden curve to the relationship. Mathematically the formula of a correlation coefficient is a formula of a straight line. However, curvilinear relationships cannot be captured in a standard correlation coefficient.

If suspected, curvilinear relationships should be investigated graphically (scatterplots)

Third variable problem

There is always a possibility that a third variable exists that correlate both with A and B and explains the correlation between them. If you only measure A and B, you will observe a correlation between them, but it does not mean that they are related directly.

Consider potential “third variables” in advance and include them in the research study to explicitly investigate the links between A, B and these “third variables”.

Spurious correlations

Spurious correlations are correlations obtained by chance. They become an issue if the research study includes multiple variables and computes multiple correlations between them. If you measure 100 correlations, there is a chance a small number will be significant, even if the variables are not related.

Results of multiple correlations should be interpreted with caution. Effect sizes need to be considered together with the level of statistical significance. Avoid formulating hypotheses after the dataset has been obtained.

Reflexivity

is the practice of critically reflecting on one's own beliefs, assumptions, and potential biases throughout the research process. It requires researchers to actively acknowledge and document their influence on data collection and interpretation.

Triangulation

involves using multiple data sources, theoretical perspectives, or researchers to cross-validate findings and reduce subjective bias.

Member checking

Member checking, a form of credibility checks, involves sharing preliminary findings with participants to ensure interpretations accurately represent their perspectives.

Audit trails

involves keeping detailed records of all research decisions, coding frameworks, and analytical choices to ensure transparency.

Rapport building

Establishing strong rapport with participants fosters open and honest communication, reducing the likelihood of altered responses due to discomfort or distrust.

Covert observations

Using covert observation techniques or passive monitoring tools prevents participants from altering their behavior due to researcher presence.

Credibility

It is a criterion used to judge the quality of qualitative research. The extent to which something or someone can be believed and trusted. The conclusions of the study must give a true picture of the phenomenon under investigation and accurately represent the perspective of the participants, that is, represent reality as the participants see it

Validity

equivalent of credibility, validity refers to the accuracy of the methods in achieving the desired aims. See internal and external, as well as construct, population, and ecological validity.

Triangulation

Triangulation allows for data gathering through multiple researchers, methods, or points in time. This offers more opportunity for replication of results and increases the robustness of findings. If the results are the conclusions will become more trustworthy/accurate and increase credibility.

· Researcher triangulation: by using two or more researchers to measure/gather the data, this can reduce research bias and/or error and increase inter-rater reliability (if the results match). As a result, the findings should be more accurate and trustworthy because the bias/errors of one researcher should be checked and corrected by the other. Researcher triangulation can occur during the recording, analysis and/or reporting of the data.

Thick/rich descriptions

In qualitative research, describing the observed behavior in sufficient detail so that it can be understood holistically and in context prevents errors by the reader to not draw assumptions if details are missing. Contextual details should be sufficient to make the description meaningful to an outsider who never observed this behavior first-hand. This can help identify and avoid potential biases and build trust with the reader.

Quantitative

Quantitative research relies on objective, numerical or measurable data. Quantitative data is analyzed statistically to identify patterns.

Qualitative

Qualitative research relies on subjective experiences, personal accounts or documents that illustrate in detail how people think or respond within society.

True lab experiment

Field experiment

Quasi-experiment

Natural experiment

Correlational study

Survey

Case study

Observation (and all of its

subcomponents)

Interview (and all of its

subcomponents)

Focus groups

Probabilistic sampling

Non-probabilistic sampling

Random sampling

Convenience sampling

Volunteer sampling

Purposive sampling

Snowball sampling

a non-probability research technique where initial participants refer additional subjects for a study, creating a growing network of participants similar to a snowball rolling down a hill.