RMS sem 1 part 2 '26

Operationalizing

turning a conceptual definition into a operational definition

How do you manage good interrater reliability with behavioral observations?

a clear codebook, that defines behavior and how to classify it to go against observer bias

What are the four types of

What is a response set and what are two types of it?

• response set; quick shortcut in a survey

• fence sitting; choosing a neutral stance

• yea-saying; agreeing with everything without thinking about it

5 types of questions in questionaires

In-depth interviewing vs focus groups

• in-depth inteviews; one-on-one, for personal topics

• focus groups; group of informants (experts on a topic) that discuss together, less personal topics

Conceptual definition vs operational definition

• conceptual; theoretical definition of a construct

• operational; how the construct is measured or manipulated

What are the three types of research claims and what do they entail?

1) Causal; one variable causes the other (manipulated)

2) Association; if one variable changes, the other does too (measured)

3) Frequency; frequency of one variable (measured)

Causal vs Association claim

• Association; measures both variables, does not require internal validity

• Causal; manipulates one variable and manipulates the other, requires internal validity

What are three criteria for a causal claim?

1) Covariance, because causal claim is a suggested explanation for an association

2) Internal validity; there should be no confounds

3) Temporal presedence; cause should come before the effect

Which research is used for every one of the three research claim types?

1) Association; correlational research

2) Causal; experimental research

3) Frequency; observational research (or survey’s)

What are the four types of validity and what do they entail?

1) Statistical; does the data back up the statistical conclusion?

2) Internal; are there no confounds?

3) External; can the effect be generalized to the population?

4) Construct; is this the right way to measure/manipulate the construct?

What are the three aspects of the statistical validity of an estimated effect?

1) strength

2) precision

3) replication

How does a manipulated variable help a study achieve temporal precedence and internal validity?

• internal validity; is there a confound?

• temporal precedence; cause comes before effect

• a manipulated variable ís the cause that comes before the effect and together with random assignment it controls for alternative explanations

What is temporal precedence and how do you achieve it?

• temporal precedence; cause comes before effect

• by manipulating a variable and measuring the effect

How do you improve the construct validity of a poll or survey?

How do you improve the construct validity of making observations?

Semantic diferrential format

What are the three types of questions in a survey or poll?

1) double-barreled; asks 2 questions in one

2) leading; encourages a certain response more than others

3) negatively worded; phrased negatively in a confusing way (e.g. double negative)

Acquiescence

yea-saying, agreeing with everything on a survey as a quick shortcut (=response set)

Faking good vs faking bad in answering a survey

• faking good; choosing a socially acceptable answer to avoid controversy

• faking bad; choosing an answer that make one look worse than they are (e.g. to get sympathy points)

What are the four types of answer formats in a survey or poll?

1) Open-ended; free to answer whatever

2) Forced-choice; having to pick between two or more answers

3) Likert; choosing on a scale from strongly agree to strongly disagree

4) Semantic differential format; choosing on a scale from contrasting adjectives

Masked design

observers are unaware of the study purposes to reduce bias (e.g. double blind study)

What is the difference between observer bias, observer effect and reactivity?

• observer bias; interpretation of behavior is influenced by expectations

• observer effect; participants behavior is influenced to match expectations by observer

• reactivity; participants behavior is influenced because they are being watched

What are two types of observation than can reduce reactivity?

• unobtrusive; being hidden (e.g. one-way mirror)

• participant; living among participants (with consent)

What are two solutions for observer bias and observer effect?

• masked design

• using multiple observers

Why is external validity essential for most frequency claims?

What are some sampling techniques that allow for a representative sample?

What are the two types of samples?

• biased; not generalizable to population

• representative; each member of the population has an equal chance of being in the sample

Propability vs non-probability sampling methods

• Probability; participants are picked at random (representative sample)

• Non-probability; participants are not picked at random (biased sample)

When is external validity a priority and when not?

Census

Testing everyone in a population

What are the 6 different types of probability sampling?

1) simple random; sample is chosen completely at random from the population

2) stratified random; identifying particular demographic categories and then randomly selecting individuals of each category

3) cluster; clusters of participants in the population are selected at random, followed by data collection of everyone in a cluster

4) systematic; counting off every Nth member of a population to achieve a sample

5) multistage; a random sample of clusters followed by a random sample of people within the selected clusters

6) oversampling; identifying particular demographic categories and then intentionally randomly overrepresenting one or more categories

Self-selection

a form of sampling bias that occurs when a sample contains only people who volunteer to participate

What are the different types of non-probability sampling?

1) purposive; only certain kinds of people are included in a sample

2) convenience; choosing a sample based on those who are easiest to access

3) snowball; participants are asked to recommend acquaintances for the study

4) quota; identifying particular demographic categories, setting a target number for each category and non-randomly sampling until the quota is full

When might researchers decide to use a non-probability sample, and when is a probability sample more important?

• Non-probability can be used; association claims, causal claims (internal validity more important than external validity)

• Probability absolutely necessary; frequency claims (external validity important)

Inferential bias

When will a larger sample not be more externally valid than a smaller one?

A large non-probability sample (non-randomly selected) reduces sampling error, not bias. A small probability sample (randomly selected) has higher external validity.

Difference between manipulated and independent variable

indepedent variable can be measured in a correlational study, but is always manipulated in an experimental study

What is the difference between a control group vs a comparison group?

• comparison group; independent variable level that differs from treatment group

• control group; independent variable level that represents a neutral condition

• Control group can be a comparison group, but not every comparison group is a control group

Matched groups

participants who are similar on some measured variable are grouped into sets; the members of each matched set are then randomly assigned to different conditions

What is a control variable and why is it used?

• control variable; variable that stays constant across conditions to improve internal validity (no confounds) (e.g. same testing room)

Independent-Groups vs Within-Groups Designs

• independent groups; both get different levels of an independent variable (e.g. 4 or 8 hours of sleep)

• within groups; both get all the levels of an independent variable (4 ánd 8 hours of sleep)

What are the two types of Indepedent-Groups designs?

• posttest-only design; dependent variable is measured only once (after manipulation of independent variable) (e.g. 2 groups drink Pepsi ór Coca Cola and rate the taste)

• pretest/posttest design; dependent variable is measured twice (before and after manipulation of independent variable) (e.g. 2 groups drink Pepsi ór Coca Cola and rate their thirst before and after)

In what case might a posttest-only design be better than a pretest/posttest design?

if the pretest makes the participants suspicious or changes their behavior

What are the two types of Within-Groups designs?

• concurrent measures; dependent variable is measured once, after exposure to all the levels of the independent variable at the same time (e.g. everyone drinks Coca Cola ánd Pepsi and chooses the best one)

• repeated measures; dependent variable is measured more than once, after exposure to each level of the independent variable (e.g. everyone drinks Coca Cola ánd Pepsi and rates the tastes)

What are the three potential threats to internal validity in an experiment?

1) design confound; the way a study is designed (manipulated) accidentally creates a confound

2) selection effect; big differences between groups

3) order effect;

When can individual differences be a confound in a study?

if individual differences are distributed unevenly (= systematic variablity) between groups

How do experimenters avoid the three threats to internal validity in an experiment?

1) design confound; experimental control variable

2) selection effect; random assignment and matched groups

3) order effect;

Systematic vs unsystematic variability

• systematic; individual differences are distributed unevenly among groups

• unsystematic; individual differences are distributed evenly among groups

Why might a within-groups design be better than an independent-groups design?

• participants are the same in both conditions, so the difference in conditions can only be attributed to the independent variable

• it requires fewer participants than an independent-groups design

Validity

the appropriateness of a conclusion or decision

What are the three types of order effects?

What are the two types of counterbalancing?

Pilot study