Paper 3 Methodology IB Psychology HL 2

generalizability

the extent to which the results of the study can be applied to a broader context beyond the sample and the settings used in the study itself.

provides numerical data that allows for statistical analysis to produce knowledge that can be applied as widely as possible

population validity

the extent to which finding can be generalized from the sample to the target population

conditions for high population validity

when a sample is representative of the target population AND an appropriate technique is used

probability sampling methods

preferred because it can reduce error and bias in sample’s selection

the method attempts to ensure every member of the target population has an equal chance of participating in the study

sample size

can influence the statistical representativeness

minimum size is 10% of the target population and should not exceed 1000

the more statistically representative the sample size the more accurately the results should reflect the potential target population

sample characteristics

sample should reflect all essential characteristics with of the target population

must identify the essential characteristics relevant to the target population BEFORE highlighting characteristics that may not have been represented well

essential sample characteristics

age, gender, ethnicity, culture, sexuality, education, language, religion

ecological validity

the extent to which findings can be generalized from the experiment to other settings and/or situations

both the task and the settings must be considered when judging for ecological validity

real-life environment

environment needs to be judged based on how realistic or artificial it is

realistic task

participants should be asked to do tasks or behaviors that they would normally do in real-life

replications

when studies are replicated and achieve the same or similar results as the original study, it gives greater validity to the findings.

quantitative research

relies on objective, numerical, and/or measurable data and is analyzed statistically to identify patters

qualitative research

relies on subjective experiences, personal accounts, and/or documents that illustrate in detail how people think and/or respond in society. this is thematically/contextually analyzed to complete and gain insights and contextual depth

experimental research key characteristics

• is manipulated and DV is measured, confounding variables are controlled, or possibilities of causal inferences

• lab or field

• distinguishable by experimental design sub-types

correlational research key characteristics

• variables are measured not manipulated

• no causation can be inferred

• refer to another way variables are measured

• defined by process not by method of data gathering

qualitative research key characteristics

• give defining characteristics to the larger method

• unique characteristic distinguishes hit from other sub-types in the same category

true lab experiment

• random allocation

• characterized by its controlled environment

• causal relationship can be established

field experiment

• naturalistic environment

• causal relationship MAY be established

• not all confounding variables may be accounted for

quasi-experiment

• IV is not manipulated but a naturally occurring characteristic of the participants

• setting can be field or natural

• no causal relationship can be established

natural experiment

• IV is not manipulated but a naturally occurring aspect that impacted participants

• extraneous/confounding variables are not always controlled

• comparison of behavior before and after the event can occur

• causal relationship can not be established

correlational study

• no IV or DV but co-variables

• measures the linear relationship between co-variables

• correlation coefficients measure -1.0 to 1 based on strength of relationship

• no causal relationship can be established

• utilized inferential statistics and statistical significance

• data is collected through surveys or questionnaires typically

survey

• gathers data, calculated correlations, and makes statistical inferences on a large number of participants

• fata gathering techniques are typically questionnaires or something similar

• often gathers a random sample

case study characteristics

• an in-depth investigation of a unique individual/group

• longitudinal

• often uses different methods

case study pros

• useful for unique phenomena

• can contradict established theories

case study cons

• researcher and participant bias

• no generalizability

• ethically confounding - hard to maintain confidentiality

observation

• researcher visually monitors participants and record their behavior

• no manipulation - no IV or DV

• limited by methods of data generation (selective attention and interpretation)

naturalistic observation

observation is done in real-life settings not arranged for the study’s purpose

naturalistic observation pros

• used to analyze behavior that is unethical to artificially simulate

• participant behavior is not impacted by research procedure

naturalistic observation cons

time consuming

laboratory observations

observation carried out in specifically designed environments and the participants know they are participating

laboratory observations pros

• possible to create rare circumstances uncommon to be naturally observed

• isolates behavior of interest efficiently

laboratory observations cons

artificial environment may influence participants behavior

overt observation

observation where participants are aware they are under observation

overt observation pros

follows ethical guidelines and obligations

overt observation cons

participant expectations due to their knowledge of observation may influence behavior

covert observations

observation where the researcher does not inform participants that they were being observed for their behavior

the researcher may or may not be actively present

covert observations pros

participants behave authentically/ have natural reactions

covert observations cons

raises ethical concerns as the researcher does not follow the ethical guidelines and obligations because participants are not aware and did not consent to this observation

participant observation

the researcher is an active member who participates in the observation group

participant observation pros

allows researcher to experience phenomena and gain personal insights

participant observation cons

risk that the researcher becomes too involved and loses objectivity

non-participant observation

researcher remains removed from the observed group

non-participant observation pros

more impartial

non-participant observation cons

some details can only be understood from the perspective of an active participant of the group

interview

• a self-report technique that has the researcher ask questions of participants to give answers on their personal experiences

• self-reporting

• give insights of the perspective of the participant

• interview data that comes in the form of of an audio/video recording is subsequently converted to an interview transcript which is analyzed to identify patterns and themes (inductive content analysis)

structured interview

• include a fixed list of questions and topics in closed questions asked in a fixed order

• highly controlled

• bound by an interview schedule

• useful when several interviewers are involved to ensure the process is standardized

structured interview pros

• does not require much training

• data is easier to analyze

• useful when several interviewers are involved to ensure the process is standardized

structured interview cons

• unique circumstances can not be accommodated in this method

• more context behind rationale of participant can not be fully explored

semi-structured interview

• uses an interview guide with a list of questions and topics that need to be covered

• allows for flexibility where elaborate answers can be provided

• questions can be open or closed

semi-structured interview pros

• fits the natural flow of a conversation better

• better suited for socially sensitive issues

• more effective in studying the unique experience of each participant

semi-structured interview cons

• less comparability across interviews as flexibility of questions are added

• interviewer must be trained and now how to establish rapport

• time consuming to analyze data

unstructured interview

• involves an informal discussion on a particular topic and the next question is determined by the participant’s prior response

unstructured interview pros

• very effective for unique cases or cases where no theoretical expectations exist that would inform the wording of questions

unstructured interview cons

• more time consuming and results are difficult to analyze and interpret

• requires extensive training

focus groups

• a special type of semi-structured interview that is simultaneously conducted with 6-10 participants

• participants are encouraged to interact with each other

• group dynamics can be analyzed

• interviewer can act as a facilitator to keep interactions focused on research questions

focus groups pros

• more natural behavior as participants interact with each other rather than just the interviewer

• interactions can reveal more than a one-on-one session

• easier to respond to sensitive questions in group settings (typically)

• multiple perspectives allow for a more holistic understanding

focus groups cons

• dominant responders can disrupt group dynamics as their assertiveness can impact and distort other participants behaviors and their responses

• more difficult to preserve and maintain confidentiality

• especially demanding in terms of sampling and creating interview transcripts

• considerably more training for a interviewer

types of probabilistic sampling research

random sampling

types of non-probabilistic sampling research

• purposive sampling

• convenience/ opportunity sampling

• snowball sampling

• volunteer/self-selected sampling techniques

probabilistic sampling technique

samples are not selected based on specific selection criteria

most rigorous approach to sampling for statistical research

non-probabilistic sampling technique

samples are selected based on specific selection criteria

random sampling

• probabilistic sampling

• when every member of the target population has an equal chance of selection

• reduce chance of sampling bias

convenience/ opportunity sampling

• non-probabilistic sampling technique

• gathers participants who happen to be available for the study for both researcher and participant

• quick, easy, and inexpensive

volunteer/self-selected sampling

• non-probabilistic sampling technique

• participants approach researchers and volunteer to be a participant

• typically a form of marketing calls for participants

• participants may have more commitment to the study since they advocated to participate

snowball sampling

• non-probabilistic sampling technique

• a group of initial participants act as a “seed” to invite more participants

• sample keeps growing until desired size has been reached

• useful when studying “hidden populations” (people who do not want others to know about them/ people generally hard to find)

purposive sampling

• non-probabilistic sampling technique

• participants are chosen for salient characteristics that are relevant to the researchers

• easy to select

• flexible as it can be supplemented with more participants need be

sampling

method of selecting a group of participants to undergo research which will create data for researchers to collect and analyze

goal of quantitative sampling

to find a sample that is statistically representative - large sample size and a chance all sub-groups of the target population are represented

goal of qualitative research

to select participants who are particularly informative about the research topics under investigation BUT generalization of findings is less important

temporal validity

an external validity that refers to the generalizability of a study’s results across time

construct validity

the extent to which results of the study can be generalized from operationalizations to theoretical constructs

when is construct validity high?

this leap is justified if the operationalization provides sufficient coverage of the construct

transferability

is the extent to which we can transfer the findings from one study to another context

case-to-case generalization

which refers to applying the findings from the setting of the research status to other setttings

how to achieve case-to-case generalization in qualitative research

• thick & rich descriptions to ensure the reader has sufficient information and details on the study’s context

• new context is similar enough to findings described in the report

• replication of findings

theoretical generalization

generalization is made from observations to the formations of a broader theory

how to achieve theoretical generalization in qualitative research

• rigorous analysis and interpretation for research findings

• data saturation

• thick, rich descriptions

• in-depth analysis

• free of bias

data saturation

further data does not add anything new to the already formulated conclusions and interpretations

sample-to-population generalization

applying the results of the study from the participants who took part in the study to the wider population

how to achieve sample-to-population generalization in qualitative research

bias

refers to factors that may negatively affect/alter the results of the study, decreasing accuracy

methodological bias/ procedural bias

flaws in the design, procedures, or materials of the study that were not avoided

how to avoid methodological bias

• methodological triangulation

• controls

• replication

methodological triangulation

using a second method to gather data to support or refute ideas in the original method

participant variability

groups are not entirely equivalent at the start of the experiment

differentiation skews the impact of/ strength of relationship

order effect/ testing effect

DV is tested twice and prior experience impacts the true results of the participants behavior because they have done it prior

instrumentation

instrument measuring the DV changes slightly between measurements, compromising standardization of the measurement process

history

outside events that happen to participant during the experiment

maturation

natural changes that participants go through in an experiment

experimental mortality

participants drop out of the experiment

only a problem if the rate of dropping out is not the same in every experimental condition

researcher bias

when thoughts, beliefs, r ideas of the researcher are negatively influencing this study’s results

ways to avoid researcher bias

• researcher triangulation

• controls

• replication

researcher triangulation

using more than one researcher in the process, reduces the chances that one researcher’s beliefs or opinions can affect the study

participant bias

when an individual’s existing thoughts, beliefs, or ideas influence their thinking or behavor

sampling bias

sample is not representative of the target population

why is quantitative research always biased?

non-probabilistic sampling techniques will introduce bias as they automatically do not equally represent the full target population.

ways to avoid participant bias

• controls

• replication

• making sure that questions are more open ended

• non-judgemental interviewers

• establish a good rapport

• keep dominant participants in check

• have an awareness of cultural differences

• participants feel safe to answer truthfully

how to avoid sampling bias

• ensure a large enough sample size OR add more participants

• recommend to attempt random sampling

demand characteristics

participants understand the true aim of the experiment and then alter their behavior intentionally or unintentionally as a result

correlational research bias

credibility is the extent to which the results of the study reflect the reality that is being investigated

bias in correlational research can occur on two levels:

• the level of variable measurement

• level of interpretation of findings