HL Paper 3

Generalisability

The extent to which the results can be applied to populations, settings, and conditions beyond those directly studied.

Generalisability in quantitative research

External validity

Population validity

How well the sample represents the wider target population.

Ecological validity

How well the findings apply to real-life settings (natural environments outside of the lab).

Generalisability in qualitative research

Representational, inferential, theoretical generalisation

Representational generalisation

How well the findings can be applied to the population from which the sample was drawn.

Inferential generalisation

How well the findings can be applied to other populations (transferability).

Theoretical generalisation

How well the findings generate theory that can be applied to other completely different contexts.

Credibility

The extent to which a research account is believable and appropriate, with particular reference to the level of agreement between participants and the researchers.

Credibility in quantitative research

Internal and external validity

Internal validity

Level of control/confounding variables, cause and effect relationship between IV and DV

Credibility in qualitative research

Triangulation (method, data, researcher, theory), rapport, iterative questioning, reflexivity (personal, epistemological), credibility checks, thick descriptions

Method triangulation

Using different data collection methods (e.g. interviews + observations) to cross-check findings.

Data triangulation

Gathering data from different people, times, or places to ensure findings aren't specific to one sample.

Researcher triangulation

Having multiple researchers analyse the data independently to avoid personal biases.

Theory triangulation

Using different theoretical perspectives to interpret the data for a more complete understanding.

Rapport

Building trust and a comfortable relationship with participants so they feel safe to give honest, detailed responses, increasing the credibility of the data collected.

Iterative questioning

Revisiting and rephrasing questions during interviews to clarify inconsistencies and deepen understanding. This helps catch lies, misunderstandings, or superficial answers.

Personal reflexivity

When a researcher thinks about how their own personal values, beliefs and experiences may have influenced their ability to be objective when carrying out and interpreting data in their study.

Epistemological reflexivity

When a researcher reflects on how the way that research was carried out may have influenced the findings.

Credibility checks

Strategies used to confirm the accuracy of findings, like participant validation (checking with participants whether the interpretations make sense to them) or peer debriefing (discussing findings with other researchers for feedback).

Thick descriptions

Providing detailed, rich contextual descriptions of participants, settings, and interactions so that readers can judge for themselves how well findings apply to other contexts ("transferability").

Bias

Human factors in the design and conduction of the study which may affect its results.

Bias in quantitative research

Single and double-blind controls, randomisation, standardised procedures

Single and double-blind controls

Participants (and researchers) do not know whether they are in the experimental/control group. Prevents participant biases e.g. demand characteristics.

Randomisation

Participants are randomly assigned to experimental conditions to help ensure that any individual differences are spread evenly across groups.

Standardised instructions and testing conditions

Ensuring all participants receive the same instructions in the same way reduces variability in how the study is administered. Consistency in the environment, materials, and procedures across all participants helps minimise bias. Precisely defining variables and how they will be measured reduces ambiguity and researcher interpretation. B

Bias in qualitative research

Avoid leading questions, reflexivity (personal and epistemological)

Avoid leading questions

Peer review of questions. Pilot interviews, then debrief interviewees to see if questions were confusing/leading. Use unstructured interview to avoid asking direct questions.

Personal reflexivity

When a researcher thinks about how their own personal values, beliefs and experiences may have influenced their ability to be objective when carrying out and interpreting data in their study.

Epistemological reflexivity

When a researcher reflects on how the way that research was carried out may have influenced the findings.

Bias in both quantitative and qualitative research

Peer review, researcher triangulation

Types of bias

Cultural, self-selection, ascertainment, focus groups

Peer review

Research should be approved by a committee of peers before being undertaken, final paper should be reviewed before publication.

Researcher triangulation

Having more than one researcher carry out interview/observation/interpret data. Inter-rater reliability.

Cultural bias

WEIRD (Western, educated, industrialised, rich, democratic) cultures and YAVIS (young, affluent, verbal, intelligent, social)

Self-selection bias

People that volunteer tend to be more highly motivated than the average person or have specific reasons why they want to be in this particular study.

Ascertainment bias

When a particular group is left out from research.

Focus groups

Dominant responder bias - one person is outspoken and contributes all the time.

Informed consent (applied/further)

Participants must be fully informed about the nature, purpose, risks, and benefits of the study and must voluntarily agree to take part.

Deception (applied/further)

Intentionally misleading or withholding information from participants about the true purpose of the study to avoid bias, but only when necessary and ethical.

Confidentiality/Anonymity (applied/further)

Confidentiality means keeping participants' data private and secure, while anonymity means that no identifying information is collected or linked to participants.

Debrief (applied/further)

After the study, participants must be given full information about the purpose, methods, and any deception used, and have any questions answered.

Right to Withdrawal (applied/further)

Participants must be informed that they can leave the study at any time without penalty or negative consequences.

Protection from harm (applied/further)

Researchers must ensure that participants are not exposed to physical or psychological harm greater than what they would normally experience in everyday life.

Informed consent or debriefing (reporting/applying)

Participants (and/or their parents) should be told how their data would be used. Participants should be told the field the results would be applied to. The ethics review board should be informed of how findings would be applied.

Right to withdrawal (reporting/applying)

Participants should be able to withdraw and remove their data from the study at any time.

Objectivity (reporting/applying)

The researcher should remain value-free and unbiased when conducting their investigations.

Reflexivity (reporting/applying)

Researchers should clarify who funded the research and the motivation for the study. They should make explicit the assumptions underlying their research and be aware of their own biases on how they may have influenced findings.

Anonymity (reporting/applying)

No participants should ever be identifiable from published research, including data.

Deception (reporting/applying)

Findings should be stated precisely and cautiously to avoid misunderstandings/misinterpretations. Researchers should directly address any misrepresentations of findings by policymakers/the media.

Beneficence (reporting/applying)

Findings should not lead to prejudicial treatment against a certain group. Not all characteristics of the sample are relevant and need to be reported.

Stigmatisation (reporting/applying)

Socially marginalised or vulnerable populations should not be described as worthy of disgrace or disapproval.

Scientific/scholarly integrity (reporting/applying)

Researchers should retract/correct errors, should not fabricate data, plagiarise, or misrepresent findings.

Sound and valid methodology (reporting/applying)

When research findings are publicised, people are likely to take them as fact and policies may be based on them. Researchers should explicitly communicate the study’s limitations.

Validity and reliability (reporting/applying)

The study should be verified, replicated, or peer reviewed before being applied.

Random (quantitative)

• Every member of the target population has an equal chance of being selected.

• Participants can be selected by drawing names from a hat or allocating everyone a number and then using a form of computerised random number generator to select the numbers.

• Should give a representative sample as the randomisation should ensure that the sample is not biased in any way.

• Should be possible to generalise from a random sample to the rest of the target population.

Convenience/opportunity (quantitative/qualitative)

• Selecting people who are readily available to participate.

• Quick, easy and convenient.

• Psychology lecturers use students in their classes to take part in their research - university students have to participate in a certain number of studies to gain credits for their degree.

• A form of non-representative sampling - it cannot be guaranteed that those who are available will match the target population.

Purposive (qualitative)

• Participants are chosen because they possess specific characteristics that make them suitable for the research project.

• Sometimes handpicked to get a more diverse set of participant for the study (maximum variation type).

• Researchers may also attempt to get a more homogenous sample - carefully select people who all have share very similar experiences and/or have similar backgrounds in order to get a more representative view of that ‘type’ of person (‘homogenous type’ sampling).

Snowball (qualitative)

• Asking people who are already part of the sample if they can think of anyone they know who meets the criteria and might agree to become part of the study.

• Particularly useful when there is no sampling frame.

• Once approached by the researcher, new participants may also give their informed consent and so the sample grows.

Volunteer (quantitative/qualitative)

• Researchers advertise for participants by placing notices, posters and adverts in places where potential participants might see them - newspapers, social media.

• Participants who are interested then get in touch with the researcher.

• People who volunteer often have a vested interest in the line of research being under-taken and often share certain similarities in terms of their personalities, life-styles, educational and socio-economic backgrounds - can lead to self selection bias and is a form of non representative sampling.

Stratified (quantitative)

• Key subgroups (or strata) within the population are first identified, then a simple random sample is taken from within each subgroup or strata

• Effective for smaller populations whose characteristics are known.

Systematic (quantitative)

• Easiest and most convenient technique for drawing a random sample.

• Selects every nth participant from a sampling frame.

Lab/true experiment

• Experimenters decide on a single variable (IV) which they manipulate/ deliberately alter to see if this brings about a change in a second variable (DV) which is measured in some quantitative way.

• All other variables which might affect the dependent variable are held constant (controlled) - any change in the DV can be said to have been caused by the IV - findings have strong internal validity.

Field experiment

• Seek causal relationships between a deliberately manipulated IV and a DV but do not take place under controlled conditions.

• Conducted in real-world environments where participants can be found going about their daily business, e.g. at a cinema or shopping centre.

• Participants are often unaware that they are being observed as part of an experiment, meaning that their behaviour is more natural.

• Increases the ecological validity of the findings, but the inability to control extraneous variables due to the natural setting decreases internal validity.

Quasi experiment

• Share some but not all of the characteristics of a laboratory experiment

• Not possible to randomly allocate participants to the experimental and/or control groups - groupings are outside of the control of the experimenter.

• Findings lack internal validity as there may be uncontrolled variables that led participants to be in one group and not the other, which are responsible for any changes in the DV.

Natural experiment

• IV is naturally occurring, i.e. the change between the experimental and control conditions is brought about as a consequence of factors which are outside of the experimenters’ control.

• Findings may lack internal validity - changes in the DV may have arisen due to extraneous factors.

Correlational studies

• Have no manipulated variable - do not seek to establish causal relationships.

• Have two or more measured variables known as co-variables which are measured using quantitative data, e.g. through some sort of rating scale.

• Common in areas of psychology where it may not be possible to ethically or practically manipulate variables as part of an experiment.

• No attempt to establish causality - internal validity relates solely to the extent to which the instruments used to measure the co-variables provide accurate and meaningful data.

Surveys

• Meant to generalise to large populations - statistical inferences can be made.

• Different to questionnaires which focus on one behaviour and can be either quantities or qualitative.

• Often use random samples - the goal is always to gain a representative sample of the target population.

• Can be used to demonstrate correlation between two variables.

Naturalistic Observation: Participant

• Observer collects data from participants in their natural environments without any deliberate manipulation of the setting.

• Quantitative data: tallying.

• Qualitative data: ‘field notes’, diagrams and sketches.

• Researcher becomes actively involved within the community or group that they are studying.

• Facilitates a unique perspective that would not have been otherwise possible - can lead to subjectivity but credibility of the data is enhanced.

Naturalistic Observation: Non-Participant

• Observer collects data from participants in their natural environments without any deliberate manipulation of the setting.

• Quantitative data: tallying.

• Qualitative data: ‘field notes’, diagrams and sketches.

• Researcher remains separate from the activities of the group and does not interact with participants whilst conducting the observation.

• Lack of involvement means that the researcher is free to focus entirely on the data collection, arguably leading to a more accurate record of events.

Case study

• Focus on a single individual, group or organisation that is unusual in some way.

• Researcher collects a detailed case history including secondary data from school reports or hospital records - allows the researcher to gain necessary insight before collecting their own primary data.

• Data gathered using a variety of techniques including interview, observation and the use of standardised tests (method triangulation) - results in a rich and detailed insight into the behaviours of interest.

Unstructured interview

• Have a clear research objective and the interview schedule will include broad topics or themes for discussion - may have a list of starter questions.

• Defining features: flexibility and freedom.

• Direction of the interview is determined by the interviewee and the interviewer bases any questions asked on their previous responses.

• Need training and experience to collect credible data - interviewers need to keep the interview on track and achieve the research objective when there is no ‘script’

• Eliciting relevant information without asking leading questions and with minimal guidance in the interview schedule requires practice and skill.

Semi-structured interview

• More flexible than a structured interview.

• Pre-determined set of questions in the interview schedule.

• Interviewer may deviate, asking follow-on questions if the interviewee presents an unanticipated and relevant view.

• Interviewer may also rephrase questions and alter the order as appropriate.

• Open questions: longer, richer answers from respondent allowing them to elaborate in their own words without being guided by the interviewer.

• Closed questions result in brief, precise answers of just a few words.

Focus group

• Comprises 8-12 people who are interviewed together about a topic of common interest - small size allows all members to have their say

• People who share similar experiences tend to bond together, developing a sense of belonging and trust which helps them to talk freely about sensitive issues.

• Researcher becomes the group facilitator and their role is to monitor the discussion, keep the group on the topic if they veer off into an irrelevant topic - must ensure that all issues raised are responded to and explored as fully as possible within the time available.

Random (quantitative)

• Every member of the target population has an equal chance of being selected.

• Participants can be selected by drawing names from a hat or allocating everyone a number and then using a form of computerised random number generator to select the numbers.

• Should give a representative sample as the randomisation should ensure that the sample is not biased in any way.

• Should be possible to generalise from a random sample to the rest of the target population.